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(54) NETWORK CONTROL SYSTEM 

(57) In a network control system, a device has 
screen display data indicative of its operating screen, 
and a controller issues a screen request (101) to obtain 
the screen display data, etc., as screen information 
(111) from the device. By using the screen display data, 
etc., the controller displays the operating screen. When 
an operation is carried out to the operating screen, its 
operation information and identification information of 

Fig. 5 



the screen display data are sent to the device as opera- 
tion request (241). The device executes a function cor- 
responding to the operation request. As a result, when 
the screen display data in the device is changed, the 
device sends the changed screen display data, etc., to 
the controller as screen information (121). 
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Description 

TECHNICAL FIELD 

[0001] The present invention relates to network 
control systems in which a device connected to a net- 
work is operated through a network and, more specifi- 
cally, a device control system with a graphical user 
interface (GUI) supporting user's device operation 
through graphics, characters, etc., on a screen. 

BACKGROUND ART 

[0002] In recent years, device control systems for 
controlling devices by selectively operating, through a 
remote controller, device control graphics (icons) com- 
posed of screen display data and characters indicating 
functions of the devices on a TV (Television) screen 
have made their appearance. Also, network systems 
with digital devices such as a DVD connected thereto 
using IEEE 1394-1995 for transmitting and receiving 
video/audio data have been emerging. 
[0003] One example of conventional network con- 
trol systems is disclosed in Japanese Patent Laying- 
Open NO.H9-149325 (1997-149325). 
[0004] This conventional network control system is 
now described below. 

[0005] Similarly to digitai interfaces according to 
IEEE 1394 standards, etc., AV units are connected to 
each other without switching by a bilateral packet com- 
munications system with a serial bus that periodically 
provides an equal communication opportunity to the 
devices. 

[0006] Here, each AV unit stores its own screen dis- 
play data. By request from a controller having a graphic 
display function (television receiver), each AV unit trans- 
mits the screen display data to the controller. The con- 
troller displays the received screen display data. The 
controller further has the functions of inquiring for data 
required for indicating the AV unit connected thereto 
and controlling the display screen based on the screen 
display data from the AV unit. Each AV unit has a 
recording medium for storing the screen display data, 
and a function of selecting the appropriate screen dis- 
play data in response to the inquiry from the controller 
for the screen display data. 

[0007] In the above structured network control sys- 
tem, the screen display data is held in each device (AV 
unit), and outputted according to a display request from 
the controller (television receiver). Graphics unique to 
each device (AV unit) are thus displayed on the screen 
of the controller. 

[0008] In the above described structure, however, 
when the display on an operating screen is to be 
changed due to internal state changes in the device, the 
device cannot report its change of state to the controller, 
thereby causing a mismatch in display screen informa- 
tion between the controller and the device. For this rea- 



son, the controller cannot correctly provide a user with 
device operation information. 
[0009] In view of the above problem, an object of 
Ihe present invention is to provide a network control sys- 

5 tern in which a device can quickly report to a controller 
about a change of the device's internal state that 
causes a change of display on an operating screen to 
ensure the controller and the device can share the 
same stale information. Such network control system is 

10 herein achieved with simple structure and small trans- 
mission load on a communication path. 

DISCLOSURE OF THE INVENTION 

15 [0010] A first aspect of the present invention is 
directed to a network control system in which a first unit 
and a second unit are connected to each other a trans- 
mission path and a controller included in the first unit 
controls a device in the second unit through the trans- 

20 mission path, 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
information, 
25 the device 

having screen display data for displaying an 
operating screen of the device and identifica- 
tion information for identifying the screen dis- 
30 play data; and 

transmitting the screen display data and the 
identification information to the controller 
through the transmission path, and 

35 the controller 

comprising a user interface including display 
means; 

receiving the screen display data and the iden- 
40 Mication information from the device through 

the transmission path; 

displaying the operating screen on the display 
means using the screen display data; and 
in response to an operation by a user to the 
45 operating screen, controlling the device by 

transmitting operation information indicative of 
the operation and the identification information 
to the device through the transmission path. 

so [0011] A second aspect of the present invention is 
directed to a network control system in which a first unit 
and a second unit are connected to each other a trans- 
mission path and a controller included in the first unit 
controls a device in the second unit through the trans- 

55 mission path, 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
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information, 
the device 

having screen display data for displaying an 
operating screen of the device and identifica- 
tion information for identifying the screen dis- 
play data; and 

transmitting the screen display data and the 
identification information to the controller 
through the transmission path; and 

the controller 

comprising a user interface including display 
means; 

receiving the screen display data and the iden- 
tification information from the device through 
the transmission path; 

dispfaying the operating screen on the display 
means using the screen display data and the 
identification information; and 
in response to an operation by a user to the 
operating screen, controlling the device by 
transmitting operation information indicative of 
the operation and the identification information 
to the device through the transmission path. 

[0012] According to the first or second aspectof the 
present invention, by identifying the screen display data 
with the Identification Information, the device can easily 
and reliably recognize to which screen display data the 
user operation information sent by the controller is 
directed, thereby preventing erroneous operation of the 
device. Further, according to the first or second aspect, 
the controller manages the screen display data repre- 
senting the operating screen of the device, and trans- 
mits the operation information based on the user's 
operation. However, the controller does not have to 
understand the meaning of the function of the device or 
the operation {the relation between the operation and 
the function of the device), in other words, user interface 
information (screen display data, operation information, 
etc.,) is managed and processed by the controller, while 
control information directly related to device control 
(intonation indicating the function of the device, intona- 
tion indicating the relation between the function of the 
device and the operation, etc.,) is by the device. There- 
fore, the processing in the controller is reduced, and the 
types of operation for controlling the device by the con- 
troller are not limited but can be varied. Furthermore, 
according to the first or second aspect, the controller 
obtains the screen display data representing the operat- 
ing screen of the device and the identification informa- 
tion from the device and display the operating screen of 
the device using the obtained data and information. 
Therefore, even if the device has a new function that 
cannot be presently assumed, only by sending the 
screen display data corresponding the new function 



from the device to the controller, the new function can 
be easily provided for and is made available to the user. 
[0013] According to a third aspect of the present 
invention, in the first or second aspect, 

5 

the identification information includes version infor- 
mation indicating a version of the screen display 
data. 

10 [0014] According to the third aspect of the present 
invention, the identification information obtained by the 
controller from the device includes the version informa- 
tion. Therefore, the controller can detect a change in the 
operating screen information of the device, and thereby 

is can easily detects that the state of the device is 
changed based on instructions from other controller or 
voluntary change in the device. Further, according to 
the user's operation, the version information of the 
screen display data representing the operating screen 

20 is sent with the operation information to the device. 
Therefore, the device can easily and reliably understand 
to which operating screen the operation is directed, 
thereby executing the operation desired by the user. 
Further, according to the third aspect, when receiving 

25 the screen display data, the controller checks the ver- 
sion information thereof, and updates display only when 
the version information is correctly updated. Therefore, 
it is possible to reduce a load on the controller when the 
same screen display data is transmitted again and 

30 again. 

[001 5] According to a fourth aspect of the present 
invention, in the first or second aspect, 

the operation information includes an operating 
35 position information indicative of an operating posi- 
tion on the operating screen. 

[001 6] According to the fourth aspect of the present 
invention, the operation information that is sent to the 

40 device according to the user's operation includes the 
operating position information indicating the operating 
position on the operating screen. Therefore, in the oper- 
ation to the operating screen, a desired operating posi- 
tion can be directly specified by a pointing device. Thus, 

45 operability for device control is improved. 

[0017] A fifth aspect of the present invention is 
directed to a network control system in which a first unit 
and a second unit are connected to each other a trans- 
mission path and a controller included in the first unit 

so controls a device in the second unit through the trans- 
mission path, 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
55 information, 
the device 

having screen display data composed of a plu- 



3 



5 



EP 1 041 851 A1 



6 



raNty of partial screen display data for display- 
ing an operating screen of the device: and 
transmitting the partial screen display data to 
the controller through the transmission path, 
and 

the controiler 

comprising a user interface including display 
means; 

receiving the partial screen display data from 
the device through the transmission path; 
displaying the operating screen on the display 
means using the partial screen display data; 
and 

in response to an operation by a user to the 
operating screen, controlling the device by 
transmitting operation information indicative of 
the operation to the device through the trans- 
mission path. 

[0018] According to the fifth aspect of the present 
invention, the screen display data can be constructed of 
a plurality of partial screen display data obtained by 
dividing the screen, thereby reducing the data amount 
to be transferred. 

[0019] According to a sixth aspect of the present 
invention, in the fifth aspect, 

when the screen display data of the device is 
changed, the device transmits changed partial 
screen display data of the screen display data to the 
controller; and 

the controller receives the changed partial screen 
display data from the device through the transmis- 
sion path and, based on the received partial screen 
display data, updates the operating screen dis- 
played on the display means. 

[0020] According to the sixth aspect of the present 
invention, when the screen display data of the device is 
changed because the state in the device is changed, for 
example, only the partial screen display data including 
the changed display element is sent to the controller. 
Therefore, the data amount to be transferred to the con- 
troiler on change of screen display is reduced, and the 
processing in the device and controller is simplified. 
[0021 ] According to a seventh aspect of the present 
invention, in the fifth aspect, 

the device 

has partial screen identification information for 
identifying the partial screen display data; and 
transmits the partial screen display data and 
the partial screen identification information to 
the controller through the transmission path, 
and 



the controller 

receives the partial screen display data and the 
partial screen identification information from 
5 the device through the transmission path. 

[0022] According to an eighth aspect of the present 
invention, in the seventh aspect, 

jo when the screen display data of the device is 
changed, the device transmits changed partial 
screen display data of the screen display data and 
the partial screen identification information of the 
partial screen display data to the controller, and 

15 the controller receives the changed partial screen 
display data and the partial screen identification 
information of the partial screen display data from 
the device through the transmission path and, 
based on the received partial screen display data 

20 and partial screen identification information, 
updates the operating screen displayed on the dis- 
play means. 

[0023] According to the eighth aspect of the present 
25 invention, when the screen display data of the device is 
changed, not only the partial screen display data includ- 
ing the change display element but also its partial 
screen identification information is sent to the controller. 
Therefore, the controller can easily specify the changed 
30 partial screen display data, thereby reducing a process- 
ing toad such as updating the display of the operating 
screen on change of the screen display data. 
[0024] According to a ninth aspect of the present 
invention, in the seventh aspect, 

35 

in response to an operation by a user to the operat- 
ing screen, the controller controls the device by 
transmitting operation information indicative of the 
operation and the partial screen identification infor- 
to mation corresponding to the operation to the device 
through the transmission path. 

[0025] According to the ninth aspect of the present 
invention, in response to an operation from the user, the 

4s operation information and the partial screen identifica- 
tion information corresponding to the operation are 
transmitted to the device. Therefore the device can eas- 
ily and reliably recognize to which partial screen display 
data the operation information transmitted to the device 

so is directed, thereby preventing erroneous operation of 
the device. 

[0026] According to a tenth aspect of the present 
invention, in the seventh or ninth aspect, 

55 the partial screen identification information includes 
version information indicating a. version of the par- 
tial screen display data. 
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[0027] According to the tenth aspect of the present 
invention, the partial screen identification information, 
which is sent to the controller together with the partial 
screen display data when the screen display data of the 
device is changed, includes version information. There- 5 
fore, the controller can prevent erroneous display of the 
operating screen by using this version information. Fur- 
thermore, in response to an operation from the user, the 
version information of the partial screen display data 
corresponding to the operation is sent to the device 10 
together with the operation information. Therefore, the 
device can easily and reliably know the operation by the 
user, thereby executing the operation as the user 
desires. 

[0028] According to an eleventh aspect of the is 
present invention, in the seventh or ninth aspect. 

one display element in screen display is arranged in 
any one of a plurality of display parts corresponding 
to the plurality of partial screen display data. so 

[0029] According to the eleventh aspect of the 
present invention, if any one of the display elements on 
the operating screen is changed, only the partial screen 
display data of that display element is sent from the 25 
device to the controller, and the plurality of partial 
screen display data do not have to be sent. Therefore, 
the data amount to be transferred to the controller on 
change of screen display is reduced, and also the 
processing in the device and controller is simplified. 30 
[0030] According to a twelfth aspect of the present 
invention, in the seventh or ninth aspect, 

a display element corresponding to each operation 
by the user is arranged in any one of a plurality of 35 
display parts corresponding to the plurality of partial 
screen display data. 

[0031] According to the twelfth aspect of the 
present invention, the display element corresponding to 40 
each operation by the user is arranged in any one of the 
plurality of display parts corresponding to the plurality of 
partial screen display data. Therefore, when, in 
response to the operation from the user, the operation 
i nf ormation and the partial screen identification i nforma- 45 
tion corresponding to that operation are sent to the 
device, the device can easily recognize the user opera- 
tion from the partial screen identification information. 
[0032] A thirteenth aspect of the present invention 
is directed to a network control system in which a first so 
unit and a second unit are connected to each other a 
transmission path and a controller included in the first 
unit controls a device in the second unit through the 
transmission path, 

55 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
information, 



the device 

having screen display data for displaying an 
operating screen of the device and overlap dis- 
play data for overlap display on the operating 
screen; and 

transmitting the screen display data and the 
overlap display data to the controller through 
the transmission path, and 

the controller 

comprising a user interface including display 
means; 

receiving the screen display data and the over- 
lap display data from the device through the 
transmission path; 

displaying the operating screen on the display 
means using the screen display data, and car- 
rying out overlap display on the operating 
screen displayed on the display means by 
using the overlap display data; and 
in response to an operation by a user to the 
operating screen, controlling the device by 
transmitting operation information indicative of 
the operation to the device through the trans- 
mission path. 

[0033] According to a fourteenth aspect of the 
present invention, in the thirteenth aspect, 

the overlap display data is cursor information for 
displaying a cursor indicative of a position of opera- 
tion by the user to said operating screen. 

[0034] According to a fifteenth aspect of the present 
invention, in the fourteenth aspect, 

the cursor information includes position information 
indicative of a position of the cursor on the operat- 
ing screen. 

[0035] According to a sixteenth aspect of the 
present invention, in the fourteenth aspect, 

the cursor information includes shape information 
indicative of a shape of the cursor. 

[0036] According to the sixteenth aspect of the 
present invention, with cursor shape information being 
set, it is possible to conform the design of the cursor to 
the design of each display element on the operating 
screen. 

[0037] According to a seventeenth aspect of the 
present invention, in the fourteenth aspect, 

the cursor information includes size information 
indicative of a size of the cursor. 
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[0038] According to the seventeenth aspect of the 
present invention, with cursor size information being 
set, it is possible to conform the size of the cursor to the 
size of each display element on the operating screen. 
[0039] According to an eighteenth aspect of the s 
present invention, in the fourteenth aspect, 

the cursor information includes color information 
indicative of a color of the cursor. 

JO 

[0040] According to the eighteenth aspect of the 
present invention, with cursor color information being 
set, it is possible to conform the design of the cursor to 
the design of each display element on the operating 
screen. Furthermore, according to the eighteenth is 
aspect, the QUI can be represented as supposed in the 
device, and thereby a GUI creator can easily convey 
his/her intention to users. 

[0041] According to a nineteenth aspect of the 
present invention, in the fourteenth aspect, 20 

the cursor information includes enable information 
indicating an operation that the user is alfowed to 
perform. 

25 

[0042] According to the nineteenth aspect of the 
present invention, when an operator carries out opera- 
tion, whether the operation is allowed or not is deter- 
mined using enable information in the cursor 
information. Therefore, if the prohibited operation is 30 
instructed, operation information is not transmitted to 
the device, and error handling can be made in the con- 
troller. 

[0043] According to a twentieth aspect of the 
present invention, in the thirteenth aspect, 35 

when a display part corresponding to the overlap 
display data of the device is changed, the device 
transmits the overlap display data to the controller, 
and 40 
the controller receives the overlap display data from 
the device through the transmission path and, 
based on the received overlap display data, 
updates the operating screen displayed on the dis- 
play means. as 

[0044] According to the twentieth aspect of the 
present invention, if only the overlap display data is 
changed due to an operation from the user, the device 
that has received the operation information indicative of so 
that operation sends overlap display data to the control- 
ler, but does not send the screen display data thereto. 
Using the received overlap display data, the controller 
updates overlap display on the operating screen. There- 
fore, when the cursor is displayed using the overlap dis- ss 
play data, for example, the data amount to be 
transferred is small, thereby improving a response 
speed to cursor move operation. 
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[0045] A twenty-first aspect of the present invention 
is directed to a network control system in which a first 
unit and a second unit are connected to each other a 
transmission path and a controller included in the first 
unit controls a device in the second unit through the 
transmission path, 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
information, 
the device 

having screen display data for displaying an 
operating screen of the device and identifica- 
tion information for identifying the screen dis- 
play data; and 

transmitting the screen display data and the 
identification information to the controller 
through the transmission path, and 

the controller 

comprising a user interface including display 
means; 

receiving the screen display data and the iden- 
tification information from the device through 
the transmission path; and 
displaying the operating screen on the display 
means using the screen display data. 

[0046] A twenty-second aspect of the present 
invention is directed to a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path and a controller included 
in the first unit controls a device in the second unit 
through the transmission path, 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
information, 
the device 

having screen display data for displaying an 
operating screen of the device and identifica- 
tion information for identifying the screen dis- 
play data; 

transmitting the screen display data and tie 
identification information to the controller 
through the transmission path, and 

the controller 

comprising a user interface including display 
means; 

receiving the screen display data and the iden- 
tification information from the device through 
the transmission path; and 
displaying the operating screen on the display 
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means using the screen display data and the 
identification information. 

[0047] A twenty-third aspect of the present inven- 
tion is directed to a network control system in which a 
first unit and a second unit are connected to each other 
a transmission path and a controller included in the first 
unit controls a device in the second unit through the 
transmission path, 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
information, 
the device 

having screen display data for displaying an 
operating screen of the device and overlap dis- 
play data for overlap display on the operating 
screen; and 

transmitting the screen display data and the 
overlap display data to the controller through 
the transmission path, and 

the controller 

comprising a user interface including display 
means; 

receiving the screen display data and the over- 
lap display data from the device through the 
transmission path; and 

displaying the operating screen on the display 
means using the screen display data, and car- 
rying out overlap display on the operating 
screen displayed on the display means by 
using the overlap display data. 

[0048] A twenty-fourth aspect of the present inven- 
tion is directed to a second unit that is connected to 
each other a transmission path to a first unit including a 
controller and includes a device controlled by the con- 
troller through the transmission path, 

at least one of the first unit and the second unit 
being for handling at feast one of video, audio, and 
information, 
the device 

having screen display data for displaying an 
operating screen of the device and identifica- 
tion information for identifying the screen dis- 
play data; 

transmitting the screen display data and the 
identification information to the controller 
through the transmission path; and 
receiving the identification information of the 
screen display data and operation information 
indicative of an operation by a user, and oper- 
ating based on the received identification infor- 



mation and operation information. 

[0049] A twenty-fifth aspect of the present invention 
is directed to a first unit that is connected to each other 
s a transmission path to a second unit and includes a con- 
troller for controlling a device included in the second unit 
through the transmission path, 

at least one of the first unit and the second unit 
10 being for handling at least one of video, audio, and 
information, 
the controller 

comprising a user interface including display 

is means: 

receiving screen display data indicative of an 
operating screen of the device and identifica- 
tion information for identifying the screen dis- 
play data from the device through the 

20 transmission path; 

displaying the operating screen on the display 
means using the screen display data; and 
in response to an operation by a user to the 
operating screen, controlling the device by 

25 transmitting operation information indicative of 

the operation and the identification information 
to the device through the transmission path. 

[0050] According to a twenty-sixth aspect of the 
30 present invention, in the twenty-fourth aspect, 

the operation information includes operating posi- 
tion information indicative of a position of operation 
on the operating screen. 

35 

[0051] According to a twenty-seventh aspect of the 
present invention, in the twenty-fifth aspect, 

the operation information includes operating posi- 
40 tion information indicative of a position of operation 
on the operating screen. 

[0052] A twenty-eighth aspect of the present inven- 
tion is directed to a second unit that is connected to 
45 each other a transmission path to a first unit including a 
controller and includes a device controlled by the con- 
troller through the transmission path, 

at least one of the first unit and the second unit 
so being for handling at least one of video, audio, and 
information, 
the device 

having screen display data composed of a plu- 
55 rality of partial screen display data for display- 

ing an operating screen of the device; 
transmitting the partial screen display data to 
the controller through the transmission path; 
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and 

receiving operation information indicative of an 
operation by a user, and operating based on 
the received operation information. 

5 

[0053] A twenty-ninth aspect of the present inven- 
tion is directed to a first unit that is connected to each 
other a transmission path to a second unit and includes 
a controller for controlling a device included in the sec- 
ond unit through the transmission path, 10 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
information, 

the controller js 

comprising a user interface including display 
means; 

receiving a plurality of partial screen display 
data indicative of an operating screen of the so 
device through the transmission path from the 
device; 

displaying the operating screen on the display 
means using the partial screen display data; 
and 25 
in response to an operation by a user to the 
operating screen, controlling the device by 
transmitting operation information indicative of 
the operation through the transmission path to 
the device. 30 

[0054] Accordi ng to a thirtieth aspect of the present 
invention, in the twenty-eighth aspect, 

when the screen display data of the device is 36 
changed, the device transmits changed partial 
screen display data of the screen display data to the 
controller. 

[0055] According to a thirty-first aspect of the m 
present invention, in the twenty-ninth aspect, 

when the screen display data of the device is 
changed, the controller receives changed partial 
screen display data of the screen display data from 4s 
the device through the transmission path and, 
based on the received partial screen display data, 
updates the operating screen displayed on the dis- 
play means. 

so 

[0056] A thirty-second aspect of the present inven- 
tion is directed to a second unit that is connected to a 
transmission path to a first unit including a controller 
and includes a device controlled by the controller 
through the transmission path, 55 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
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information, and 
the device 

having screen display data for displaying an 
operating screen of the device and overlap dis- 
play data for overlap display on the operating 
screen; 

transmitting the screen display data and the 

overlap display data to the controller through 

the transmission path; and 

receiving operation information indicating a 

user's operation transmitted from the controller, 

and operating based on the received operation 

information. 

[0057] A thirty-third aspect of the present invention 
is directed to a first unit that is connected through a 
transmission path to a second unit and includes a con- 
troller for controlling a device included in the second unit 
through the transmission path. 

at least one of the first unit and the second unit 
being for handling at least one of video, audio, and 
information, 
the controller 

comprising a user interface including display 
means; 

receiving screen display data indicative of an 
operating screen of the device and overlap dis- 
play data for overlap display on the operating 
screen through the transmission path; 
displaying the operating screen of the device 
on the display means using the screen display 
data, and performing overlap display on the 
operating screen displayed on the display 
means using the overlap display data; and 
in response to an operation by a user to the 
operating screen, controlling the device by 
transmitting operation information indicating 
the operation through the transmission path to 
the device. 

[0058] A thirty-fourth aspect of the present inven- 
tion is directed to a control method, in a network control 
system in which a first unit and a second unit are con- 
nected to each other a transmission path, at least one of 
the first unit and the second unit being for handling at 
least one of video, audio, and information, and a control- 
ler included in the first unit controlling a device included 
in the second unit through the transmission path, the 
control method comprising the steps of: 

transmitting screen display data for displaying an 
operating screen of the device and identification 
information for identifying the screen display data 
from the device Ihrough the transmission path to 
the controller; 
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displaying the operating screen on the controller 
using the screen display data transmitted from the 
device; and 

in response to an operation by a user to the operat- 
ing screen, controlling the device by transmitting 
operation information indicative of the operation 
and the identification information from the controller 
through the transmission path to the device. 

[0059] A thirty-fifth aspect of the present invention 
is directed to a control method, in a network control sys- 
tem in which a first unil and a second unit are connected 
to each other a transmission path, at least one of the 
first unit and the second unit being for handling at least 
one of video, audio, and information, and a controller 
included in the first unit controlling a device included in 
the second unit through the transmission path, the con- 
trol method comprising the steps of: 

transmitting screen display data for displaying an 
operating screen of the device and identification 
information for identifying the screen display data 
from the device through the transmission path to 
the controller; 

displaying the operating screen on the controller 
using the screen display data and the identification 
information transmitted from the device; and 
in response to an operation by a user to the operat- 
ing screen, controlling the device by transmitting 
operation Information indicative of the operation 
and the identification information from the controller 
through the transmission path to the device. 

[0060] A thirty-sixth aspect of the present invention 
is directed to a control method, in a network control sys- 
tem in which a first unit and a second unit are connected 
to each other a transmission path, at least one of the 
first unit and the second unit being for handling at least 
one of video, audio, and information, and a controller 
included in the first unit controlling a device included in 
the second unit through the transmission path, the con- 
trol method comprising the steps of: 

transmitting at least one of plurality of partial screen 
display data composing screen display data for dis- 
playing an operating screen of the device from the 
device through the transmission path to the control- 
ler; 

displaying the operating screen on the controller 
using the partial screen display data transmitted 
from the device; and 

in response to an operation by a user to the operat- 
ing screen, controlling the device by transmitting 
operation information indicative of the operation 
from the controller through the transmission path to 
the device. 

[0061 ] A thirty-seventh aspect of the present inven- 



tion is directed to a control method, in a network control 
system in which a first unit and a second unit are con- 
nected to each other a transmission path, at least one of 
the first unit and the second unit being for handling at 
5 least one of video, audio, and information, and a control- 
ler included in the first unit controlling a device included 
in the second unit through the transmission path, the 
control method comprising the steps of: 

10 transmitting screen display data for displaying an 
operating screen of the device and overlap display 
data for overlap display on the operating screen 
from the device through the transmission path to 
the controller; 

15 displaying the operating screen on the controller 
and performing overlap display on the displayed 
operating screen based on the overlap display data 
by using the screen display data and the overlap 
display data transmitted from the device; and 

20 in response to an operation by a user to the operat- 
ing screen, controlling the device by transmitting 
operation information indicative of the operation 
from the controller through the transmission path to 
the device. 

25 

[0062] A thirty-eighth aspect of the present inven- 
tion is directed to a control method, in a network control 
system in which a first unit and a second unit are con- 
nected to each other a transmission path, at least one of 
30 the first unit and the second unit being for handling at 
least one of video, audio, and information, and a control- 
ler included in the first unit controlling a device included 
in the second unit through the transmission path, said 
control method comprising the steps of: 

35 

transmitting screen display data for displaying an 
operating screen of the device and identification 
information for identifying the screen display data 
from the device through the transmission path to 
40 the controller; and 

displaying the operating screen on the controller 
using the screen display data transmitted from the 
device. 

45 [0063] A thirty-ninth aspect of the present invention 
is directed to a control method, in a network control sys- 
tem in which a first unit and a second unit are connected 
to each other a transmission path, at least one of the 
first unit and the second unit being for handling at least 

so one of video, audio, and information, and a controller 
included in the first unit controlling a device included in 
the second unit through the transmission path, the con- 
trol method comprising the steps of: 

55 transmitting screen display data for displaying an 
operating screen of the device and identification 
information for identifying the screen display data 
from the device through the transmission path to 
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the controller; and 

displaying the operating screen on the controller 
using the screen display data and the identification 
information transmitted from the device. 

5 

[0064] A fortieth aspect of the present invention is 
directed to a control method, in a network control sys- 
tem in which a first unit and a second unit are connected 
to each other a transmission path, at least one of the 
first unit and Ihe second unit being for handling at least io 
one of video, audio, and information, and a controller 
included in the first unit controlling a device included in 
the second unit through the transmission path, the con- 
trol method comprising the steps of: 

15 

transmitting screen display data for displaying an 
operating screen of the device and overlap display 
data for overlap display on the operating screen 
from the device through the transmission path to 
the controller; and 20 
displaying the operating screen on the controller 
and performing overlap display on the operating 
screen based on the overlap display data by using 
the screen display data and the overlap display 
data transmitted from the device. 25 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0065] 

30 

FIG. 1 is a block diagram showing the configuration 
of a device in a network control system according to 
a first embodiment of the present invention. 
FIG. 2 is a block diagram showing the configuration 
of a controller in the network control system accord- 35 
ing to the first embodiment. 
FIG. 3 is a diagram showing one example of system 
configuration of the network control system in the 
first embodiment. 

FIG. 4 is a diagram showing the structure of screen 40 
information sent from a device to a controller in the 
first embodiment. 

FIG. 5 is a diagram illustrating one protocol of the 
network control system in the first embodiment. 
FIG. 6 is a diagram showing one example of an 45 
operating screen displayed on the controller in the 
first embodiment. 

FIG. 7 is a diagram illustrating another protocol of 
the network control system in the first embodiment. 
FIG. 8 is a diagram showing the structure of screen so 
information sent from a device to a controller in a 
second embodiment of the present invention. 
FIG. 9 is a diagram showing one example of an 
operaling screen displayed on the controller in the 
second embodiment. 55 
FIG. 10 is a block diagram showing the configura- 
tion of a version manager of the device in the sec- 
ond embodiment. 



FIG. 1 1 is a flowchart showing the operation of the 
version manager of the device in the second 
embodiment. 

FIG. 12 is a diagram illustrating version information 
in the second embodiment. 
FIG. 13 is a diagram illustrating a protocol of a net- 
work control system in the second embodiment. 
FIG. 14 is a diagram showing one example of an 
operating screen displayed on the controller in a 
third embodiment of the present invention. 
FIG. 15 is a diagram illustrating a protocol of a net- 
work control system in the third embodiment. 
FIG. 16 is a diagram shewing the structure of 
screen information sent from a device to a control- 
ler in a fourth embodiment of the present invention. 
FIG. 17 is a diagram showing one example of an 
operating screen displayed on the controller in the 
fourth embodiment. 

FIG. 18 is a diagram showing one example of an 
operating screen displayed on the controller in a 
modified fourth embodiment. 
FIG. 19 is a diagram showing the structure of 
screen information sent from a device to a control- 
ler in a fifth embodiment of the present invention. 
FIG. 20 is a diagram showing one example of an 
operating screen displayed on the controller in the 
fifth embodiment. 

BEST MODE FOR CARRYING OUT THE INVENTION 

[0066] The present invention is now described in 
more detail referring to the attached drawings. 

(First embodiment) 

[0067] FIG. 1 is a block diagram showing the config- 
uration of a device in a network control system accord- 
ing to a first embodiment of the present invention, and 
FIG. 2 is a block diagram showing the configuration of a 
controller in the network control system according to the 
first embodiment. The operation and configuration of 
the network system of the present embodiment are now 
described by using FIGS. 1 and 2. 
[0068] A device described herein means to be con- 
trolled by a controller. The controller herein means that 
it has a user interface including display means such as 
a video monitor having a graphic display function and 
operating means such as a mouse, keyboard, and TV 
remote and controller that a user operates while watch- 
ing the screen of the video monitor. Note that either one 
or both of the device and the controller may exist in a 
single unit Further, the device corresponds to one node 
on a transmission path, and a unit may be constructed 
so as to have a plurality of nodes in a single box. The 
controller and the video monitor may be different units, 
and the connection therebetween is arbitrarily deter- 
mined: they may be connected in an analog manner or 
with a transmission path as shown in the present 
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embodiment. 

(1.1 Device in the first embodiment) 

[0069] As shown in FIG. 1, the device in the net- 
work control system of the present embodiment 
includes a packet transmitter/receiver 2, a synchronous 
data transmitter/receiver 3. a device signal processing 
part 4, an asynchronous data transmitter/receiver 5, a 
device asynchronous data processing part 6, a unit con- 
figuration information 7, an information manager 18, 
and a screen display data generator 20. The packet 
transmitter/receiver 2 is connected to a transmission 
path 1 . 

[0070] The transmission path 1 is, by way of exam- 
pie only, a serial bus defined by IEEE 1394 standard 
(IEEE 1394-1995 and standards upward-compatible 
therewith) (1394 bus). Note that the transmission path 1 
is not necessarily the 1394 bus, but is implemented by 
using ATM, Ethernet, infrared transmission, or Ihe like. 
[0071] The packet transmitter/receiver 2 physically 
and electrically interfaces with the transmission path 1 , 
and also carries out arbitration of bus mastership, cycle 
control for synchronous transfer, and other processing. 
Further, the packet transmitter/receiver 2 selectively 
receives a packet on the transmission path 1 according 
to the destination and transmits the packet to the trans- 
mission path 1 . 

[0072] At transmission, the synchronous data trans- 
mitter/receiver 3 manages the transfer rate (data divid- 
ing) and provides a header. For example, when the 
1394 bus of the AV protocol (EC 61883) standard is 
used, the synchronous data transmitter/receiver 3 pro- 
vides a CIP (Common Isochronous Packet) header. On 
the other hand, when receiving data, the synchronous 
data transmitter/receiver 3 rearranges received packets 
in a correct order, removes the headers, and performs 
other processing. 

[0073] The device signal processing part 4 receives 
synchronous data from the synchronous data transmit- 
ter/receiver 3 to carry out signal processing according to 
the device. For example, if the device is a 
recorder/player such as a digital VCR, the device signal 
processing part 4 records the synchronous data into a 
recording medium (magnetic tape, for example), in 
addition, the device signal processing part 4 retrieves 
the synchronous data from the recording medium, 
broadcasting wave, etc., and transmits the same to the 
synchronous data transmitter/receiver 3. 
[0074] The asynchronous data transmitter/receiver 
5 carries out asynchronous data transactions according 
to the protocol of the transmission path 1 . By way of 
example, for the 1394 bus, read, write, lock transactions 
are performed. Here, the asynchronous data transmit- 
ter/receiver 5 may be implemented as software. 
[0075] The device asynchronous data processing 
part 6 processes the asynchronous data received from 
the asynchronous data transmitter/receiver 5, and then 



transfers the same to the appropriate component in the 
device. For example, if receiving a control data and user 
operation information as the asynchronous data, the 
device asynchronous data processing part 6 deter- 

s mines validity of the asynchronous data, and instructs a 
unit internal controller 9 to execute functions corre- 
sponding to the control code and user operation infor- 
mation if the data is valid. Here, the device 
asynchronous data processing part 6 may transfer the 

w control code and the user operation information to the 
information manager 18. in this case, the information 
manager 18 determines validity of these and, if valid, 
instructs the unit internal controller 9 to execute func- 
tions indicated by these. 

is [0076] If the controller requests for the unit configu- 
ration information 7, the device asynchronous data 
processing part 6 sends, according to the request 
received through the asynchronous data transmit- 
ter/receiver 5, etc., the information in the unit configura- 
te tion information 7 through the asynchronous data 
transmitter/receiver 5, etc., to the controller. Further- 
more, based on the instruction from the unit internal 
controller 9, the device asynchronous data processing 
part 6 sends asynchronous data from the components 

ss in the device to the asynchronous data transmit- 
ter/receiver 5. Here, the asynchronous data transmit- 
ter/receiver 5 and the device asynchronous data 
processing part 6 may be constructed as a single unit. 
[0077] The unit configuration information 7 indi- 

30 cates information about the configuration of the device, 
being written according to the rule shown by a configu- 
ration ROM of a CSR (Command and Status Registers) 
architecture indicated by ISO/IEC 13213:1994. If the 
1394 bus is used, the device configuration information 

35 includes bus information such as whether the device 
supports a bus manager and isochronous operation , 
unit directory including information whether the device 
supports an AV protocol and a unique ID as an identifier 
of the device. 

40 [0078] When transmitting bitmap data indicative of 
contents to be displayed on the screen of the display 
means of the controller as a device operating screen 
(hereinafter referred to as "screen display data") to the 
controller, the information manager 18 provides the 

45 screen display data with identification information for 
identifying the screen display data. Further, the informa- 
tion manager 18 instructs the screen display data gen- 
erator 20 to update the screen display data according to 
the instruction from the unit internal controller 9. 

so [0079] This identification information includes infor- 
mation for specifying the device (unique ID to the 
device, for example) and version information. By way of 
example, when the identification information includes 
version information, the information manager 18 is con- 

55 structed using a counter to manage the version of the 
screen display data. The counter in the information 
manager 18 is incremented every time the screen infor- 
mation is updated by the unit internal controller 9. Here, 
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the counter is an infinite cyclic counter having a suffi- 
cient finite bit length, and indicates a minimum value 
after its maximum value is incremented. The bit length 
of the counter is arbitrarily determined. Preferably, how- 
ever, the bit length is sufficient enough to prevent one 5 
cycle of the counter value from being complete at least 
while the controller holds control or has access to a 
change of state in order to avoid the same counter value 
from indicating different screen display data. 
[0080] Further, when the screen display data is 10 
read to the controller, the counter value is also read 
thereto. The counter value indicates the version infor- 
mation of the read screen display data. Thus, the con- 
troller can confirm the version information, thereby 
improving reliability. is 
[0081 ] The screen display data generator 20 gener- 
ates screen display data (screen display data in a bit- 
map format, for example) indicating the operating 
screen for the device according to the instruction from 
the unit internal controller 9 based on the state of the 20 
device and user operation. Here, the screen display 
data generated by the screen display data generator 20 
is screen display data displayed on one screen of the 
controller. The size of the operating screen is predeter- 
mined through the communication between the device ss 
and the controller, standards, or the like, or written in the 
header of the screen information, etc. On the operating 
screen displayed using the screen display data, control 
information for controlling the device, contents informa- 
tion, operating state information, etc., are displayed. 30 
[0082] Here, the control information indicates an 
operating panel (reverse button, stop button, record but- 
ton, etc.,) of a VCR, for example. The contents informa- 
tion indicates information for notifying the user of the 
contents. For an STB (set-top box), the contents infor- 35 
mation indicates information about a program now 
being broadcast, that is, a program title, title image, 
theme music, summary, cast, etc. For a DVD device, the 
contents information indicates the contents recorded on 
a disk as a recording medium, that is, a title, title image, 40 
theme music, summary, cast, etc. For the VCR, by way 
of example only, the operating state information indi- 
cates a display element (also called "object") such as 
display elements indicating the state of operation of the 
device (playing, reversing, unattended recording pro- 45 
grammed). Further, information required for network 
control such as identification information of the control- 
ler that is now using the device, and time, data, and a 
channel number for unattended recording, etc., may be 
written in the operating state information. so 
[0083] The unit infernal controller 9 controls each 
components including the internal mechanism of the 
device, etc. If the device asynchronous data processing 
part 6 receives control code indicating an operation of 
the device, the unit internal controller 9 causes the 55 
device to operate according to the control code as 
instructed by the device asynchronous data processing 
part 6. Also, according to a change of state in the device 
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and the user operation, the unit internal controller 9 
instructs the screen display data generator 20 to update 
the screen display data through Ihe information man- 
ager 18. 

[0084] The operation of the device in response to 
the request from the controller, etc., is as follows. If the 
device or controller is connected to the transmission 
path 1 , the controller first reads the unit configuration 
information 7 of the device to determine whether the 
device has screen display data or not, making a request 
to the device for GUI information and reading the screen 
display data from the screen display data generator 20. 
[0085] Then, if receiving the user operation infor- 
mation from the controller, the device carries out 
processing indicated by the operation information 
according to the circumstances. 
[0086] Here, for example, when the controller trans- 
mits operating information indicating that the user per- 
forms "select" operation (hereinafter referred to as 
"operation information "select"") while displaying the 
operating screen for the device using the screen display 
data, the asynchronous data transmitter/receiver 5 
reports the operation to the unit internal controller 9. 
The unit internal controller 9 provide an instruction to 
each component in the device according to the state of 
the screen display data. By way of example, assume 
that screen display data indicating that the cursor is dis- 
played on the play button is transmitted from the device 
to the controller and the user performs "select" opera- 
tion while the controller displays the screen display 
data, in this case, the device receives the user opera- 
tion information "select", and the unit internal controller 
9 recognizes, from the screen display data or the state 
of the device that has generated the screen display 
data, that the user operation indicates "play", and pro- 
vides an instruction to each components in the device to 
start the play operation. 

[0087] Note that, at this time, in addition to the user 
operation information, the identification information of 
the screen display data being displayed by the controller 
when the user performed operation is transmitted to the 
device. Therefore, the device can correctly determine to 
which screen display data the user operation is 
directed. Such determination can be easily made even 
if the user operation becomes invalid or indicates con- 
trol of another function due to a change of state in the 
device (for example, when the screen display data is 
totally changed). Thus erroneous operation of the 
device can be prevented. As such, if the user operation 
becomes invalid due to a change of information in the 
device, the device can update the screen display data 
and notify the user that the operation becomes invalid. 
[0088] As described above, in response to the 
request from the controller for GUI (Graphical User 
Interface) information, all the device has to do is to pro- 
vide the screen display data information, thereby light- 
ening the load on the device. Further, it is not necessary 
to define a command to each function of the device by 
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standardization organizations etc. Even for a device 
having a new function that cannot be presently 
assumed, if screen display data compliant with the new 
function is sent to the controller, the new function can be 
easily presented to the user via the transmission path 1 , 
and become available to the user. Then, by specifying 
the function of the device using the user operation infor- 
mation and the screen display data, the user can control 
the device with a simple structure. 
[0089] Note that the component such as the syn- 
chronous data transmitter/receiver 3 and the device sig- 
nal processing part 4 may be arbitrarily implemented 
according to the function of the device, and may be 
omitted. Furthermore, such component may be imple- 
mented as either hardware or software. 
[0090] Further, the user operation information may 
be encoded and then sent. Moreover, the screen infor- 
mation and the user operation information are not 
required to be sent or received through the same trans- 
mission path. By way of example, the screen informa- 
tion is transmitted through a 1394 bus using a cable 
while the operation information by infrared rays. 

< 1 .2 The controller in the first embodiment) 

[0091 ] F IG. 2 is a block diagram showing the config- 
uration of the controller in the network control system of 
the present embodiment. The controller includes a con- 
troller signal processing part 10, a controller asynchro- 
nous data processing part 11, a screen display data 
manager 12, a display/function selection part 14, and a 
screen display data holding part 21. Note that, in FIG. 2, 
the same components as those shown in FIG. 1 are pro- 
vided with the same reference numerals, and their 
description is omitted herein. 

[0092] The controller signal processing part 10 
receives synchronous data from the synchronous data 
transmitter/receiver 3, and carries out signal processing 
that is capable of being performed by the controller. By 
way of example, if the controller is capable of displaying 
video such as a video monitor, the controller signal 
processing part 10 decodes the synchronous data 
(MPEG2 stream, for example) for display on the screen. 
[0093] The controller asynchronous data process- 
ing part 1 1 processes asynchronous data received from 
the asynchronous data transmitter/receiver 5, and then 
transmits the processed data to each appropriate com- 
ponent in the controller. Furthermore, the controller 
asynchronous data processing part 1 1 receives, from 
the packet transmitter/receiver 2 through the asynchro- 
nous data transmitter/receiver 5, information about the 
device on the transmission path 1 such as connection of 
a new device and removal of an existing device, and 
screen display data of the device, and then transmits 
them to the screen display data manager 12. Further, 
according to the instruction from the display/function 
selection part 14, the controller asynchronous data 
processing part 1 1 sends asynchronous data from each 



component in the controller to the asynchronous data 
transmitter/receiver 5. Here, the asynchronous data 
transmitter/receiver 5 and the controller asynchronous 
data processing part 1 1 may be constructed as a single 
s unit 

[0094] Assume the controller and the device are 
constructed in a single unit. Since the screen display 
data of the device in the unit is generated in the unit 
internal controller 9, the unit has the screen display data 

10 of the device but the screen display data manager 12 
does not necessarily required to manage it. Also, in this 
case, the controller signal processing part 10 and the 
device signal processing part 4, and the controller asyn- 
chronous data processing part 1 1 and the device asyn- 

15 chronous data processing part 6 may be constructed as 
a single unit, respectively. 

[0095] The screen display data manager 12 man- 
ages screen display data information received from the 
device on the transmission path 1. When receiving, 

so from the controller asynchronous data processing part 
11, information about new device connection, the 
screen display data manager 12 instructs the controller 
asynchronous data processing part 11 to read screen 
display data information of the new device. Note that, in 

25 this case, the new device may search for the controller 
to spontaneously transmit its screen display data to the 
controller. 

[0096] Next, when the screen display data and its 
version information of the new device has been read, 

30 the screen display data is held in the screen display 
data holding part 21 , and its identification information is 
stored in the screen display data manager 12 in relation 
to the screen display data. Here, the identification infor- 
mation includes the read version information. The iden- 

35 tification information may be stored in the screen display 
data holding part 21 together with the screen display 
data, or may be stored and managed by the screen dis- 
play data manager 1 2. Furthermore, when, for example, 
information about removal of an existing device on the 

40 transmission path 1 is received, the corresponding 
screen display data is deleted from the screen display 
data holding part 21. Note lhat the screen display data 
in the controller does not have to be exactly the same in 
format as that in the device as long as they include the 

45 same information. 

[0097] The display/function selection part 14 noti- 
fies the user of the screen display data including a dis- 
play element (video/audio/character information, etc.,) 
indicating a function or state of the device on the screen 

so of the controller and, according to user operation, 
selects a device and function, and provides instructions 
of executing each function. This display/function selec- 
tion part 14 instructs the screen display data manager 
12 to search for the screen display data and, using the 

55 screen display data obtained through search, causes 
the operating screen for an arbitrarily device on the 
transmission path 1 to be displayed on the screen of the 
controller. Further, the display/function selection part 1 4 
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can display/reptay data received from the controller sig- 
nal processing part 10 (video and audio data, for exam- 
ple) and data received from the controller asynchronous 
data processing part 1 1 . At this time, the screen display 
data may be overlaid on the video data received from 5 
the controller signal processing part 10, or the display 
screens for the screen display data and for the video 
data may be switched according to a user's instruction, 
etc. Further, the background of the screen display data 
may be transparent to allow display of the video data. w 
[0098] If the user performs, onto the screen dis- 
played by the controller, operation such as pressing a 
selection button on the remote controller and pressing a 
right-direction arrow button, the display/function selec- 
tion part 14 issues the identification information of the is 
screen display data and operation information indicative 
of this user operation through the controller asynchro- 
nous data processing part 1 1 , etc. 
[0099] The display/function selection part 14 
receives, as updated screen display data, a response so 
from the device to the identification information and the 
user operation information through the controller asyn- 
chronous data processing part 11. The identification 
information is managed by the screen display data man- 
ager 12, and the corresponding screen display data in 2S 
the screen display data holding part 21 is updated. Fur- 
ther, if the identification information added to the screen 
display data as the response includes version informa- 
tion, the screen display data manager 12 checks the 
version information, and updates the screen display so 
data held in the screen display data holding part 21 only 
when the version information has been correctly 
updated. Thus, the screen display data held in the con- 
troller does not have to be updated every time the same 
screen display data is transmitted, thereby lightening a 35 
load. 

[0100] In the present embodiment, the controller 
does not have to recognize each function of the device. 
For example, even for a device having a new function 
that cannot be presently assumed, the controller can 40 
notify the user by displaying screen display data includ- 
ing a display element (button, for example) to the new 
function. When, with such display element, the user 
understands and selects the new function, the dis- 
play/function selection part 1 4 of the controller transmits 45 
this user operation information to the device to cause 
the device to execute the new function. Therefore, with 
the above structure, the user can execute even a new 
function that cannot be presently assumed. 
[0101] Note that the components such as the syn- so 
chronous data transmitter/receiver 3 and the controller 
signal processing part 10 may be omitted according to 
the function of the controller. 

< 1 .3 System structure in the first embodiment ) 55 

[0102] FIG. 3 is a diagram showing the system 
structure of the network control system of the present 



embodiment. As shown in FIG. 3, the network control 
system includes a television receiver (television) 21, a 
television remote controller 22, a personal computer 
(PC) 23, a recordabie/replayable DVD 31, a DV-format 
digital VCR (DVC) 32, a VHS-formal digital VCR 
(DVHS) 33, a DV-format digital movie (DVC movie) 34, 
and a set-top box (STB) for CS digital broadcasting 35, 
which are collectively called video/audio/information 
devices. 

[0103] These video/audio/information devices are 
connected to the transmission path 1 to form a single 
network control system (such system is called "AVC 
system".) Note that the video/audio/information devices 
are not restricted to the above mentioned devices, but 
include all of existing devices in video, audio, and infor- 
mation areas (printer and mini-disc, for example) and 
devices coming in the future. 
[01 04] In such system of the present embodiment, 
the television 21 is a unit composed of a controller and 
a device, and the user uses the remote controller 22 to 
provide instructions to the display/function selection 
part 14. The PC 23 is a unit composed of a controller 
and a device (modem that interfaces with a telephone 
line, video monitor, etc.,) and the user uses a keyboard, 
mouse, etc. to provide instructions to the display/func- 
tion selection part 14. 

[0105] Here, the DVD 31 and the DVC movie 34 are 
units capable of recording and replaying audio and 
video data (hereinafter referred to "AV data"). Each of 
the DVC 32 and the DVHS 33 is a unit composed of a 
device capable of recording and replaying AV data and 
a device having a function as a digital broadcasting 
tuner. The STB 35 is a device having a function as a 
tuner for receiving CS digital broadcasting. 
[0106] Here, the DVD 31, DVC 32, DVHS 33, DVC 
movie 34, and STB 35 are all devices. However, each of 
these units may be implemented as a unit including a 
controller and a device as long as such unit can realize, 
if small at all, an environment enabling the user to oper- 
ate other devices through a liquid crystal panel, etc., 
that is, for example, as long as the unit enables the user 
to carry out operation such as selecting functions of 
other devices with a touch panel, remote controller, etc. 
Each of these units may include a processing function 
as a controller with its remote controller. In such case, 
only display and audio may be displayed on a monitor 
through analog connection, etc., and the user may oper- 
ate the unit through the remote controller while watching 
the displayed screen. At this time, the unit may include 
a controller and a device. 

( 1 .4 Screen display data in the first embodiment ) 

[0107] FIGS. 4(a) to (c) are diagrams showing the 
structure of screen display information to be transmitted 
from the device to the controller in the present embodi- 
ment. FIG. 4(a) shows screen information using an ID 
as the identification information; FIG. 4(b) shows screen 
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information whose identification information includes 
version information; and FiG. 4(c) shows screen infor- 
mation using version information as the identification 
information. 

[0108] First, the screen information shown in FiG. 
4(a) has an identifier (ID) for identifying each screen dis- 
play data. As such, by identifying the screen display 
data with the identifier, the device can easily and reliably 
recognize to which screen display data the user opera- 
tion information from the controller to the device is 
directed, thereby preventing erroneous operation of the 
device. 

[0109] The screen information shown in FIG. 4 (b) 
has, as the identification information, an identifier (ID) 
for identifying each screen display data and version 
information. For example, if the device is a VCR and 
each of a deck part and a tuner part has a different 
operating screen, the device can be constructed as 
such that each operating screen is provided with an 
identifier, and version information according to changes 
in each operating screen. In this way, if the screen dis- 
play data is identified by the identifier and the version 
information, the device can manage by classification 
and reliably recognize to which screen display data the 
user operation information from the controller to the 
device is directed, thereby preventing erroneous opera- 
tion of the device. Furthermore, the device can easily 
manage the screen display data. Still further, even if the 
device resent the same screen display data due to prob- 
lems in the transmission path, etc., the controller can 
easily know the fact and does not have to carry out 
unnecessary screen update, thereby reducing the 
processing. 

[0110] The screen information shown in FIG. 4(c) 
does not have an identifier (ID) for identifying each 
screen display data, but uses version information 
instead as the identification information. As such, by 
using a value which sequentially changes according to 
the changes of the screen display data as the identifica- 
tion information, the controller and the device can easily 
manage to which screen display data the user operation 
information from the controller to the device is directed. 
The device can thus easily know which screen display 
data the controller is now displaying and how much dif- 
ference there is between the screen on the controller 
and the screen display data of the device. Further, if the 
screen display data transmitted from the device is miss- 
ing or required to be resent due to problems on the 
transmission path, etc., the controller can easily know 
the fact of missing or a retransmission cycle to change 
the processing according to the degree of missing or 
control the screen display in consideration of the 
retransmission cycle, thereby providing a more usable 
display screen to the user. 

[01 1 1 ] Note that the screen information may include 
in its header part type information indicating a type of 
the screen display data (for example, a format of the bit- 
map data), attribute information indicating whether the 



screen display data is split screen display data or not, or 
has version information or not, size information indicat- 
ing the size of the screen display data, and the like. 
[0112] Further, the physical/logical structure of the 

5 screen information which was read into the controller 
does not have to be the same in format as that in the 
device as long as information that is valid for the screen 
display included in the screen information in the control- 
ler is the same as that in the device. In other words, the 

io screen information does not have to physically include 
the header as shown in FIGS. 4(a) to (c), etc., as long 
as information such as header is managed by the con- 
troller. 

is { 1 .5 Control operation in the first embodiment) 

[0113] FIG. 5 is a diagram illustrating one protocol 
of the network control system in the first embodiment. 
The operation for controlling the controller and the 

20 device is now described below using the drawing. 
[0114] First, if a device 200 is newly connected to 
the transmission path 1, a controller 100 on the trans- 
mission path 1 recognizes the new device 200 through 
bus reset if the bus is a 1394 bus, for example. On the 

25 instruction for the screen display data manager 12, the 
controller first issues a screen request 101 through the 
transmission path 1 to the device 200. Responding to 
the screen request 101, the device 200 sends the 
screen display data and its identification information as 

so screen information 111, and the controiier 100 then 
reads the screen information 111. Then, the screen dis- 
play data manager 12 of the controiier 100 reads and 
registers the screen display data to store it in the screen 
display data holding part 21 of the controller 100. 

35 [0115] Here, the structure may be as such that, 
when the new device 200 is connected to the transmis- 
sion path 1 , the device 200 spontaneously transmits the 
screen display data to the controller 1 00. Alternatively, 
the structure may be as such that, when the new device 

40 200 is connected to the transmission path 1 , the control- 
ler 100 and the device 200 communicate with each 
other to determine a transmission method of the screen 
display data and the device 200 transmits the screen 
display data to the controller 100 by following the 

45 method. 

[0116] The screen display data read into the con- 
troller 100 is identified by the unique ID for each device 
200 and the identification information. For example, the 
screen display data has a list structure or directory 
so structure with an entry for each device 200. Note that 
how the screen display data is held is arbitrarily deter- 
mined. 

[01 1 7J The display/function selection part 1 4 of the 
controller 100 refers to the screen display data in the 
ss screen display data holding part 21 through the screen 
display data manager 12, and displays the same on the 
display screen. Here, if the user desires to display the 
operating screen for another device 200 that is con- 
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nected to the controller 100. for example, the screen 
display data of the devices 200 that are connected to 
the controller 100 are sequentially displayed on the dis- 
play screen by using a display switching button on the 
remote controller, and then the screen display data rep- 5 
resenting the operating screen for the desired device 
200 is displayed. 

[0118] Here, if the controller 100 displays the 
screen display data of a specific device 200, the dis- 
play/function selection part 14 makes a request to the 10 
screen display data manager 12 for the screen display 
data of that device 200 using the unique ID of the device 
200, etc. The screen display data manager 12 retrieves 
the screen display data of the device 200 from all screen 
display data in the screen display data holding part 21 , is 
and then transmits the same to the display/function 
selection part 14. The display/function selection part 14 
causes the screen display data to be displayed on the 
display screen. 

[0119] However, if the device 200 connected to the 20 
transmission path 1 is included in a device list of the dis- 
play/function selection part 14 listing connected 
devices, but the screen display data of that device 200 
is not in the screen display data holding part 21, the 
screen display data manager 12 sends the screen 25 
request 101 shown in FIG. 5 to the device 200 as a 
command. In response, the device 200 returns the 
screen information 111. The screen response 111 
includes the screen display data indicative of the oper- 
ating screen of the device 200, etc., and the identifica- 30 
tion information of this screen display data. The 
identification information includes the identifier (ID), the 
version information, etc. This version information indi- 
cates the version of the screen display data, and is 
updated every time the screen is updated. 35 
[01 20] Note that the device list is a list that is made 
in the display/function selection part 14 by searching the 
transmission path 1 according to change information 
about the devices 200 connected to the transmission 
path 1 (a bus reset signal, for example, for reporting 40 
addition/removal) and detecting the connected devices 
200. 

[0121] The device 200 may return only the identifi- 
cation information in the device 200 as the screen infor- 
mation 1 11 . At this time, the controller 1 00 has to further 45 
read the screen display data body from the device 200. 
However, if the controller 100 has cached the screen 
display data, for example, in order to transmit the same 
screen as that previously transmitted, the device 200 
does not have to send the screen display data body to so 
the controller 100, thereby reducing data amount to be 
transmitted, lightening a load on the transmission path, 
and simplifying the processing in the controller 100 and 
the device 200. 

[01 22] In this way, the screen display data manager ss 
12 reads the screen display data of the device 200 
thereinto. Therefore, the display/function selection part 
14 uses the screen display data manager 1 2 to read out 



the desired screen display data from the screen display 
data holding part in the controller 100, and causes the 
same to be displayed on the screen. Thus, the screen 
display data of the arbitrary device 200 can be displayed 
on the screen. 

[01 23] Next, if the user operates a button on the 
remote controller, etc., the display/function selection 
part 14 transmits the type of the operated button, the 
operation ("press", "release", "select", for example), and 
the identification information of the screen display data 
to the device 200. In other words, assuming that the 
remote controller has arrow keys indicating upward, 
downward, righlward, and leftward directions, if the user 
presses the rightward direction arrow key, for example, 
the controller 100 transmits to the device 200, as an 
operation request 241, operation information ("right- 
ward arrow key" and "press") and the identification infor- 
mation of the screen display data obtained from the 
device to the device 200. Therefore, the device 200 can 
easily and reliably recognize to which operating screen 
ihe operation is directed, and therefor can execute cor- 
rect operation. If the device 200 is a pointing device 200 
such as a mouse is used for the operating screen dis- 
played on the controller 100, the operating information 
transmitted to the device 200 includes information indi- 
cating the operating position of the mouse, etc (herein- 
after referred to as "operating position information"). In 
this case, the device 200 recognizes, from the transmit- 
ted identification information and operating Information 
including the operating position information, which oper- 
ation has been performed to which operating screen, 
and then executes operation corresponding to the rec- 
ognized results. With such structure as controlling the 
device 200 through operation of the mouse, etc.. the 
cursor can be directly moved to a desired position on 
the operating screen, thereby improving operability. 
[01 24] Note that only the operation information may 
be transmitted to the device 200 as the operation 
request 241 . In this case, although the device 200 can- 
not reliably recognize to which operating screen the 
operation is directed, the processing can be simplified 
and the traffic on the transmission path can be reduced. 
Here, the user operalion information may be encoded 
and then transmitted with the identification information. 
Alternatively, each user operation information may be 
sent as a single command (its operand is the control 
code such as the ID of the screen display data). Fur- 
thermore, if a button allowing only selecting operalion to 
the screen display data is pressed, for example, the 
structure may be as such that only the control code indi- 
cating this button is transmitted to the device 200, 
thereby simplifying the processing and reducing traffic 
on the transmission path 1 . 

[0125] Next, if the state in the device 200 is 
changed to cause the screen display data of the device 
200 to be changed, the device 200 spontaneously 
returns screen information 121. The screen information 
121 includes the identification information and the 
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screen display data, and its structure is similar to that 
shown in FIGS. 4{a) to (c). Note that the device 200 may 
be structured so that the screen information 121 
becomes a secondary response to the screen request 
101. At this time, the controller 100 receives the sec- 
ondary response, and then again issues a screen 
request. Then, if the screen display data is not changed, 
a primary response to the screen request after the sec- 
ondary response may include only the identification 
information without the screen information. With such 
structure, the correspondence between the request and 
command can be clarified, and the processing of the 
device 200 can be simplified. 
[0126] The screen information 121 is used, by way 
of example only, for the device 200 to change the dis- 
play of the operating screen into a display indicative of 
rewinding if automatic fast-rewinding has started 
because a tape comes to the end after replay in a VCR. 
Alternatively, the screen information may be used, by 
way of example only, for changing a still picture on the 
operating screen (from a convex display to a concave 
display) according to the operation such as press- 
ing/releasing a button on the operating screen. 
[0127] Afterthe screen display data in the controller 
100 is updated, the screen display data manager 12 
instructs the display/function selection part 14 to update 
the screen display. The display/function selection part 
14 updates the screen display. 
[0128] Note that, in addition to each 
request/response, etc., as shown in the present embod- 
iment, other communications may be made between the 
controller 100 and the device 200. For example, 
although not shown, the structure may be as such that 
an acknowledge signal for acknowledging each request 
and response is returned. With such procedure, the 
controller 100 can always recognize the state of the 
device 200. 

[0129] FIG. 6 is showing one example of the operat- 
ing screen displayed on the controller in the first embod- 
iment. In this example, a screen display 500 
represented by the screen display data transmitted from 
the device is displayed as the operating screen of the 
VCR. Displayed on the screen display 500 are a display 
element indicating that the device to be controlled is the 
VCR; two title images as contents information; reverse, 
play, fast-forward, pause, stop, and record buttons as 
control information; and channel up and down buttons. 
[0130] In the protocol shown in FIG. 5. the opera- 
tion information and the identification information of the 
screen display data are sent from the controller 100 to 
the device 200 as the operation request. Alternatively, 
as shown in FIG. 7, the version information may be used 
instead of the identification information in the operation 
request 241. With this version information, the device 
200 can determined to which operating screen the oper- 
ation is directed, thereby obtaining the effect similar to 
that in a case where the protocol shown in FIG. 5 is 
used. 



[0131] In the present embodiment, assuming that 
the version information of the screen display data is sent 
from the device to the controller as part of the screen 
information as shown in FIGS. 4(b) and (c), a change of 

5 state information of the device can be reported to the 
controller by using the version information, and to which 
screen display data the operation information from the 
controller is directed can be easily determined. There- 
fore, the device can change its own state information at 

10 an arbitrary time. In other words, even between the time 
the controller detects a change in the state information 
of the device and the time the controller takes in the 
stale information, the device may change its own state 
information. Therefore, the device is not required to use 

is a buffer for temporarily storing the change in the state 
information, thereby simplifying the processing and 
reducing its storage area. 

[01 32] Note that the present inventi on is not applied 
only when the screen display data of the device is 

20 updated, but can also be applied when the controller 
keeps track of arbitrary screen display data of the 
device. By way of example only, the present system is 
effective also for a monitoring system for monitoring for 
intruders with images changed and a system for keep- 

25 ing track of the state of a device operating at a remote 
site. 

( 1 .6 Effects of the first embodiment) 

30 [0133] As described above, according to the 
present embodiment, the device has screen display 
data representing an operating screen of the device and 
identification information of the screen display data. The 
controller reads the screen display data and the identif i- 

35 cation information from the device. If the identification 
information includes version information, a change of 
the operating screen information of the device can be 
detected by the identification information. Therefore, the 
controller can easily detect that the state of the device 

40 has been changed according to an instruction from 
another controller or a change made spontaneously by 
the device itself. Also, the controller can easily and reli- 
ably identify the operation information in which the state 
of the device has been correctly reflected. Moreover, the 

45 device can easily and reliably recognize to which oper- 
ating screen the operation is directed, thereby executing 
the operation as the user desires. 
[0134] Further, according to the present embodi- 
ment, the controller manages the screen display data 

so representing the operating screen for the device, and 
transmits the operation information to the device based 
on the operation by the user. However, the controller 
does not have to understand the meaning of the func- 
tion of the device or the operation (the relation between 

55 the operation and the function of the device). In other 
words, user interface information (screen display data, 
operation information, etc.,) is managed and processed 
by the controller, while control information directly 
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related to device control {information indicating the 
function of the device, information indicating the relation 
between the function of the device and the operation, 
etc.,) is by the device. Therefore, according to the 
present embodiment, the processing in the controller is s 
reduced. 

<2. Second embodiment) 

[0135] Next, a network control system according to 10 
a second embodiment of the present invention is 
described. The structures of the device and controller in 
the present embodiment are similar to those in the first 
embodiment, and therefore the same components are 
provided with the same reference numerals, and their 15 
description is omitted herein (refer to FIGS. 1 and 2). 

(2.1 Screen information in the second embodiment) 

[0136] FIG. 8 is a diagram showing the structure of 20 
screen information sent from the device to the controller 
in the present embodiment. In the present embodiment, 
the screen information is constructed of a plurality of 
partial screen information corresponding to partial 
screen display data obtained by dividing screen display ss 
data indicative of an operating screen into two or more. 
Each partial screen information is in a list format, includ- 
ing various header information and attribute informa- 
tion. However, such partial screen information does not 
have to be in a list structure. Alternatively, each partial 30 
screen information may be independent, or a screen 
information set list does not have to be required. 
[0137] Screen information 53 shown in FIG. 8 is 
constructed of a screen information set list 70, and par- 
tial screen information 61 , 62, and 63. The structure of as 
a header in the screen information is similar to that in 
the first embodiment. The screen information set list 70 
is constructed of an identifier corresponding to the 
screen information 53 and a plurality of menu entries. In 
each menu entry, an identifier for each partial screen 40 
display data in the partial screen information 61 to 63 in 
the screen information 53 is set. 
[0138] Each partial screen information 61 to 63 is 
constructed of a header, version information, arrange- 
ment information, and the partial screen display data. 45 
Here, the partial screen display data is bitmap data for 
displaying a partial screen that composes part of an 
operating screen corresponding to the screen informa- 
tion 53. The header and version information are similar 
to those in the first embodiment. The arrangement infor- so 
mation is information for specifying where the partial 
screen display data is placed on the operating screen 
corresponding to the screen information 53. 
[0139] FIG. 9 is a diagram showing one example of 
the operating screen displayed on the controller in the ss 
present embodiment. In this example, the operating 
screen is constructed of four partial screen displays 
501 , 502, 503. and 504. Therefore, the screen display 



data representing the operating screen is constructed of 
four partial screen display data corresponding to these 
four partial screen displays. As shown in FIG. 9, in the 
present embodiment, the screen display data is divided 
into a plurality of partial screen display data in such a 
manner that a display element indicative of a single 
meaning to the user is not divided. 

(2.2 Version management in the second embodiment) 

[0140] FIG. 10 is a block diagram showing the con- 
figuration of a part in the information manager 18 of the 
device carrying out management of a version of the 
screen display data (hereinafter referred to as "version 
manager"). The version manager in the present embod- 
iment is constructed of a screen display version genera- 
tor 91 , an update screen display determination part 92. 
and a plurality of partial screen display version informa- 
tion holding parts 95, 96, 97 

[0141] In the above constructed version manager, 
the screen display version information generator 91 
manages the version of the screen display data gener- 
ated by the screen display data generator 20. The 
screen display version information generator 91 is 
implemented by using a counter, and obtains screen 
display update information from the unit internal control- 
ler 9 every time the screen display data generated by 
the screen display data generator 20 is changed by the 
information manager 18 to increment the counter. The 
counter is an infinite cyclic counter having a sufficient 
finite bit length, and indicates a minimum value after 
incremented to be the maximum. The bit length of the 
counter is arbitrarily determined. Preferably, however, 
the bit length is sufficient enough to prevent one cycle of 
the counter value from being complete at least while the 
controller holds control or has access to a change of 
state in order to avoid the same counter value from indi- 
cating different screen display data. 
[0142] The update screen display determination 
part 92 uses the screen display update information 
obtained from the unit internal controller 9 to detect 
which partial screen display data has been changed, 
and caused the partial screen display version informa- 
tion holding part corresponding to the partial screen dis- 
play data to store version information of the updated 
partial screen display data. 

[0143] The first partial screen display version infor- 
mation holding part 95 holds version information of any 
partial screen display data composing the screen dis- 
play data indicative of the operating screen for the 
device. The version information of this partial screen 
display data (hereinafter referred to as "first partial 
screen display data") is, for example, version informa- 
tion of the partial screen display data in the partial 
screen information 61 shown in FIG. 8, that is, the ver- 
sion information constructing the partial screen informa- 
tion 61. 

[0144] The second partial screen display version 
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information holding part 96 holds version information of 
any partial screen display data, but other than the above 
first partial screen display data, composing the screen 
display data indicative of the operating screen for the 
device. The version information of a second partial 
screen display data is, for example, version information 
of the partial screen display data in the partial screen 
information 62 shown in FIG. 8, that is, the version infor- 
mation constructing the partial screen information 62. 
[0145] The third partial screen display version infor- 
mation holding part 97 holds version information of any 
partial screen display data, but other than the above first 
and second partial screen display data, composing the 
screen display data indicative of the operating screen 
for the device. The version information of a third partial 
screen display data is, for example, version information 
of the partial screen display data in the partial screen 
information 63 shown in FIG. 8, that is, the version infor- 
mation constructing the partial screen information 63. 
[0146] FIG. 1 1 is a flowchart showing the operation 
of the version manager in the information manager 1 8 of 
the device. In the version manager, when the device is 
activated, the screen display version information gener- 
ator 91 first initializes the version information of all par- 
tial screen display data, for example, setting the version 
information to "0" (step S701). Next, the version man- 
ager waits until the screen display data is changed (step 
S702). When the screen display data is changed, the 
screen display version information generator 91 incre- 
ments the counter to update the version information of 
the screen display data (step S703). The incremented 
counter value indicates a screen display version 
number after update. Next, the update screen display 
determination part 92 detects using the screen display 
update information which partial screen display data 
has been changed, and then causes the partial screen 
display version information holding part concerned with 
the changed partial screen display data to update the 
version information thereof. That is, the screen display 
version number is substituted into the version informa- 
tion of the partial screen display data, which is the coun- 
ter value incremented of the screen display version 
information generator 91 (step S704). In this way. after 
the version information of the partial screen display data 
is updated, it is determined whether an instruction for 
ending this series of processing by the version manager 
is provided or not. If no, the procedure returns to step 
S702 to repeat the above series of processing (steps 
S702 to S705). While the device 200 is powered on, the 
procedure repeats the above series of processing. If it is 
determined yes (step S705), the series of processing 
ends. 

[0147] By way of example only, when the second 
partial screen display data is changed, the above ver- 
sion manager operates as follows. First, the screen dis- 
play version information generator 91 obtains screen 
display update information indicative of the change of 
the second partial screen display data from the unit 



internal controller 9 to increment the counter of the 
screen display version information generator 91, 
thereby updating the version information of the screen 
display data. The update screen display determination 

5 part 92 detects by this screen display update informa- 
tion that the changed display element belongs to the 
second partial screen display data, and causes the sec- 
ond partial screen display version information holding 
part 96 to update the version information of the partial 

w screen display data. 

[0148] FIG. 12 is a diagram illustrating how the 
screen display data and the version information of each 
partial screen display data change. Here, assume that 
the screen display data indicative of the operating 

15 screen of the device is constructed of first to third partial 
screen display data. 

[0149] At the initial state, all items of the version 
information have been initialized. That is, the screen 
display version number and each partial screen display 

so version number are set to "0". Here, if a display element 
of the second partial screen display data is changed, 
the counter of the screen display version information 
generator 91 is incremented, thereby setting the screen 
display version number and a second partial screen dis- 

25 play version number to "1 ". Then, if a display element of 
the third partial screen display data is changed, the 
above counter is updated as such that the screen dis- 
play version number and a third partial screen display 
version number are both set to "2". At this time, the sec- 

30 ond partial screen display version number still indicates 
"1". Then, if a display element of the second partial 
screen display data is changed again, the above coun- 
ter is further incremented as such that the screen dis- 
play version number and the second partial screen 

35 display version number are both set to "3". In other 
words, the second partial screen display version 
number has not and will not become "2". 
[0150] In Ihis way, if any display element is 
changed, the counter of the screen display version infor- 

40 mation generator 91 is incremented, and the screen dis- 
play version number and the version number of the 
partial screen display data to which the changed display 
element belongs are both updated to take the counter 
value after increment 

45 

(2.3 Control operation in the second embodiment) 

[01 51 ] FIG. 1 3 is a diagram illustrating a protocol of 
the network control system in the second embodiment. 

so Using the drawing, the control operation of the controller 
and device is now described below. Note that the oper- 
ation other than the control operation described below is 
similar to that in the first embodiment. 
[0152] When the device 200 is newly connected to 

55 the transmission path 1 , the controller 1 00 on this trans- 
mission path 1 issues, like the first embodiment, the 
screen request 101 to the new device 200. In response, 
the new device 200 sends screen information nib, and 
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the controller 100 reads the screen information there- 
into. At this time, the screen information 111b sent from 
the device 200 is the screen information 53 including 
the screen information set list 70 shown in FIG, 8. How- 
ever, without using the screen information set list 70, all 5 
partial screen information may be sent as the screen 
information 111b. 

[01 53] Then , wh en the user carries out operati on to 
the device 200 through the operating screen for the con- 
troller 100 as a user interface, the controller 100 trans- 10 
mits the identification information of the screen display 
data indicative of the operating screen and the opera- 
tion information indicative of the operation to the device 
200 as the operation request 241, like in the first 
embodiment. At this time, the identification information w 
may include version information. 
[0154] Then, if the state in the device 200 is 
changed and thereby the screen display data in the 
device 200 is changed, the device 200 spontaneously 
transmits partial screen information 121b to the control- 20 
ler 100. Unlike the screen information 121 transmitted 
from the device 200 to the controller 100 in the first 
embodiment, this partial screen information 121b is 
composed of the partial screen display data to which a 
changed display element belongs and its identification 25 
information. That is, in the present embodiment, if the 
screen display data is changed, the partial screen dis- 
play data to which the changed display element belongs 
is transmitted to the controller 1 00. In the controller 1 00, 
after receiving this partial screen display information so 
121b, the screen display data manager 12 updates, 
among the partial screen display data held in the screen 
display data holding part 21, the partial screen display 
data indicated by the identification information of the 
partial screen information 121b with the partial screen 35 
display data of the partial screen information 121b. In 
addition to the partial screen display data, the version 
information held in the screen display data holding part 
21 is also updated. 

40 

{2.4 Effects of the second embodiment) 

[0155] In the present embodiment, as shown in 
FIG. 9, the screen display data is divided into a plurality 
of partial screen display data in such a manner that a 45 
display element indicative of a single meaning to the 
user is not divided. Therefore, if only a single element is 
changed, the plurality of the partial screen display data 
are not required to be sent. Thus, the data amount to be 
transferred to the controller when the screen display is so 
changed is reduced, and the processing in the device 
and the controller is simplified. 
[0156] Furthermore, according to the present 
embodiment, an identifier is provided for each partial 
screen display data. If the screen display data is 55 
changed, the partial screen display data to which a 
changed display element belongs and its identifier are 
sent to the controller. Therefore, the changed partial 



screen display data can be easily specified. Moreover, 
at this time, the version information of the partial screen 
display data is also transmitted to the controller. With 
this version information, the controller can prevent erro- 
neous display. 

[0157] Note that a manner of dividing the screen 
display data may varied every time the device changes 
the screen display data according to the change of the 
information in the device, By way of example, the con- 
troller always reconstructs the partial screen display 
data transmitted from the device to create a frame of 
screen display data. The created screen display data is 
stored in screen display data holding part 21 . Then, the 
partial screen display data updated by the device is 
superimposed on the screen display data held in the 
screen display data holding part 21 for update. With 
such structure, the device can transmit partial screen 
display data of an arbitrary size at an arbitrary position 
to the controller, thereby simplifying the transmission 
amount and the processing of the controller and device. 
Moreover, at this time, in response to the user's opera- 
tion, the controller sends the version information of the 
screen display data together with the user operation 
information to the device. 

(3. Third embodiment) 

[01 58] Next, a network control system according to 
a third embodiment of the present invention is now 
described. Since the structure of the device and control- 
ler in the present embodiment is similar to that in the 
first embodiment, the same components are provided 
with the same reference numerals, and their description 
is omitted herein (refer to FIGS. 1 and 2). 
[0159] Furthermore, the structure of the screen 
information in the present embodiment is similar to that 
in the second embodiment (refer to FIG. 8) in that the 
screen display data indicative of the operating screen of 
the device is composed of a plurality of partial screen 
display data. However, the present embodiment is dif- 
ferent from the second embodiment in the manner of 
dividing the screen display data into a plurality of partial 
screen display data. 

[0160] FIG. 14 is a diagram showing one example 
of the operating screen displayed on the display screen 
of the controller in the present embodiment. In the 
present embodiment, the partial screen display data is 
generated for one or more display elements selected 
through a single operation by the user, that is, for every 
operation. The screen display data is constructed of this 
partial screen display data generated in the above 
described manner, and partial screen display data rep- 
resenting a display which is a base for the display ele- 
ments indicated by the generated partial screen display 
data. For example, the operating screen shown in FIG. 
14 is constructed of a plurality of partial screen displays 
511 to 520 and partial screen display 510 which is a 
base for these partial screen displays 511 to 520 (here- 
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inafter referred to as "base screen display"). Therefore, 
the screen display data indicative of the operating 
screen of FIG. 1 4 is composed of the partial screen dis- 
play data corresponding to these partial screen display 
51 1 to 520. Each display element as an operation unit 5 
that corresponds to each of the plurality of partial 
screen displays 511 to 520 is superimposed onto the 
partial screen display 510, that is, the base screen dis- 
play for overlap display. 

[0161] FIG. 15 is a diagram illustrating a protocol of 10 
the network control system in the present embodiment. 
By using the drawing, the control operation of the con- 
troller and device is now described below. Note that the 
operation other than the control operation described 
below is similar to that in the second embodiment. 15 
[0162] The controller 1 00 issues the screen request 
101 to the device 200. In response, the device 200 
transmits the screen information 111b to the controller 
100. Then, when the user carries out operation to the 
device 200 through the user interface of the controller so 
100, the controller 100 issues an operation request 
241c to the device 200. By issuing the operation request 
241c, the controller 100 sends identification information 
of the partial screen display data to which a display ele- 
ment to be operated by the user belongs and operation 2s 
information indicative of the operation to the device 200. 
in other words, as to the operation request 241c in the 
present embodiment, the identification information of 
the partial screen display data corresponding to the 
operation is transmitted to the device 200, Instead of the so 
identification information of the screen display data 
indicative of the operating screen. In this point, the 
present embodiment is different from the second 
embodiment. In the present embodiment, as shown in 
FIG. 1 4, each operation unit displayed on the operating 35 
screen has a one-to-one correspondence with each 
partial screen display data. Therefore, by receiving the 
operation request 241b, the device 200 can easily rec- 
ognize the user operation included in the operation 
request 241b. In other words, in the device 200, based 40 
on the identification information of the partial screen dis- 
play data and the operation information received as the 
operation request 241b. the device asynchronous data 
processing part 6 instructs the unit internal controller 9 
to execute a function that corresponds to the operation 45 
unit indicated by the identification information and the 
operation information. The unit internal controllers exe- 
cutes the function based on the instruction. 
[0163] Then, when the state in the device 200 is 
changed and the screen display data in the device 200 so 
is thereby changed, the device 200 spontaneously 
transmits partial screen information 121b to the control- 
ler 100. The partial screen information 121b is com- 
posed of, similarly to the second embodiment, the 
partial screen display data to which a changed display 55 
element belongs and its identification information. After 
receiving the partial screen information 121b, the con- 
troller 1 00 updates, similarly to the second embodiment, 



the partial screen display data and its version informa- 
tion. 

[0164] According to the present embodiment as 
described above, the screen display data indicative of 
the operating screen is divided for every operation unit. 
Therefore, the device 200 can easily recognize the 
user's operation by the identification information of the 
partial screen display data transmitted from the control- 
ler 100 as the operation request 241c. Note that, if the 
partial screen data is divided in such a manner that any 
operation unit is not divided, the device 200 can easily 
recognize the operation even if division is made not by 
each operation unit. This is because, although the oper- 
ation itself cannot necessarily be specified only with the 
identification information of the partial screen display 
data, the type of operation can be narrowed down with 
the identification information. 
[0165] Further, in the present embodiment, when 
one display element is changed, the device 200 trans- 
mits to the controller 100 only the partial screen infor- 
mation corresponding to the partial screen display data 
to which the display element belongs. Therefore, as in 
the second embodiment, the data amount to be trans- 
ferred to the controller 100 with change in screen dis- 
play can be reduced. 

[0166] Still further, the controller by itself can also 
make the cursor indicative of the operating position to 
the user displayed. Therefore, the controller can man- 
age cursor movement, thereby simplifying the process- 
ing in the device, reducing the transmission amount, 
and improving response to the user's operation. 

<4. Fourth embodiment) 

[0167] Next, a network control system according to 
a fourth embodiment of the present invention is now 
described. The structure of the device and controller in 
the present embodiment is also similar to that in the first 
embodiment. Therefore, the same components are pro- 
vided with the same reference numerals, and their 
description is omitted herein (refer to FIGS. 1 and 2). 
[0168] FIG. 16 is a diagram showingthe structure of 
screen information 54 sent from the device to the con- 
troller in the fourth embodiment of the present invention. 
The screen information 54 is constructed of a screen 
information set list 71, main screen information 65, and 
cursor display element information 80. In the present 
embodiment, a cursor display element indicative of the 
operating position of the user is superimposed on the 
operating screen for overlap display. The operating 
screen itself corresponds to the main screen informa- 
tion 65, while the cursor display element to the cursor 
display element information 80. As shown in FIG. 16, 
the main screen information 65 is composed of screen 
display data as bitmap data indicative of the operating 
screen, a header including an identifier of the screen 
display data, and version information of the screen dis- 
play data. The cursor display element information 80 is 
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composed of cursor data as bitmap data representing 
the cursor display element, a header including an iden- 
tifier of the cursor data, version information of the cursor 
data, and position information indicative of the position 
of the cursor. The screen information set list 71 is com- 5 
posed of a header including an identifier of the screen 
display data indicative of the operating screen, in a 
broad sense, including the cursor display element 
(broad-sense screen display data), a data entry to the 
main screen information 65, and a data entry to the cur- 10 
sor display element. The identifier of the screen display 
data indicative 0! the operating screen itself is set as the 
data entry to the main screen information 65, while the 
identifier of the cursor data is set as the data entry to the 
cursor display element. (5 
[0169] FiG. 17 shows one example of the operating 
screen displayed on the display screen of the controller 
in the present embodiment. In this example, a cursor 
display element 600 is superimposed onto a screen dis- 
play 500, which is the operating screen itself, for overlap 20 
display. Note that it can be thought that the broad 
screen display data indicative of the broad operating 
screen that includes the cursor display element 600 is 
composed of two pieces of partial screen display data 
corresponding to the screen display 500 and the cursor 2s 
display element 600. Furthermore, the cursor position 
on the operating screen is represented by coordinate 
values (X and Y coordinates) taking an upper-left corner 
of the screen display 500 as the origin. 
[01 70] The protocol of the network control system in so 
the present embodiment is basically similar to that in the 
second embodiment as shown in FIG. 1 3. in the present 
embodiment, the screen information 111b transmitted 
from the device 200 to the controller 100 as a response 
to the screen request 101 is constructed as shown in 35 
FIG. 16. The main screen information 65 and the cursor 
display element information 80 correspond to different 
partial screen display information in the second embod- 
iment. After receiving such screen information, the con- 
troller 100 displays the operating screen using the 40 
screen display data of the main screen information 65 
and, by using the cursor data and position information of 
the cursor display element information 80, further 
superimposes the cursor display element onto the oper- 
ating screen for overlap display. 4s 
[0171] If the controller 100 issues the operation 
request 241 , the operation information is transmitted to 
the device 200 together with the identification informa- 
tion of the screen display data. The operation informa- 
tion includes the operating position information so 
indicative of the operating position of the cursor. The 
operating position information is composed of X and Y 
coordinates on the operating screen. 
[0172] After issuance of the operation request 241 , 
when the state in the device 200 is changed and there- 55 
fore the screen display data in the device 200 is 
changed, the device 200 spontaneously transmits the 
partial screen information 121b to the controller 100, 



like in the second embodiment. At this time, in the 
present embodiment, as to cursor movement, only the 
cursor display element information 80 shown in FIG. 16 
is sent to the controller 100. After receiving, the control- 
ler 100 updates overlap display of the cursor display 
element on the operating screen based on the position 
information included in the cursor display element infor- 
mation 80. Therefore, according to the present embodi- 
ment, the response speed to cursor move operation is 
improved. 

[0173] Note that the bitmap data of the cursor may 
be transmitted only in the first response (screen infor- 
mation 1 1 1 b) to the screen request, and screen infor- 
mation transmitted thereafter does not have to include 
the bitmap data of the cursor. Therefore, the processing 
can be simplified, the transmission amount can be 
reduced, and the response speed can be improved. 
Furthermore, the structure may be as such that the bit- 
map data indicative of the cursor is sent not from the 
device, but determined by the controller itself. At this 
time, as to cursor movement, etc., the device transmits 
only the updated cursor position information as screen 
information, thereby simplifying the processing, reduc- 
ing the transmission amount, and improving the 
response speed. 

(4.1 Modified example of the fourth embodiment) 

[0174] In the above embodiment, the operating 
screen itself is represented by single screen display 
data. However, as in the second embodiment, the oper- 
ating screen may be represented by a plurality of partial 
display screen data. FiG. 18 is a diagram showing one 
example of display of the operating screen in the 
present embodiment represented by a plurality of partial 
screen display data. In this example, the operating 
screen is composed of four partial screen displays 501 , 
502, 503, and 504, and a cursor display element 601 is 
superimposed onto this operating screen for display. 
Therefore, in this case, the main screen information 65 
in the screen information 54 shown in FIG. 16 is 
replaced by a plurality of partial screen display informa- 
tion as shown in FIG. 8. Note that the cursor display ele- 
ment information can be considered as one type of 
partial screen display information. 
[0175] In this case, the structure of the cursor dis- 
play element information is similar to that of the cursor 
display element information 80 shown in FIG. 16. This 
information in the cursor element information is repre- 
sented by the coordinates on the operating screen in 
the example shown in FIG. 16. In this case, however, as 
shown in FIG. 18, the arrangement information is infor- 
mation indicating a relative position on the partial 
screen display 502 onto which the cursor display ele- 
ment 601 is superimposed for overlap display. That is, 
ihe cursor position information is represented by coordi- 
nate values (relative X and Y coordinates) taking the 
upper-left corner of the partial screen display 502 as the 
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origin. 

[0176] As described above, the protocol of trie net- 
work control system in a case where the screen display 
data is composed of a plurality of partial screen display 
data is basically simitar to that in the third embodiment 
as shown in FIG. 15. in this case, when the user carries 
out operation through the cursor, the controller 100 
issues the operation request 241 c to the device 200. At 
that time, the identification information of the partial 
screen display data sent to the device 200 includes the 
identification information of the cursor data. For exam- 
ple, in the case shown in FIG. 1 8. in addition to the iden- 
tification information of the partial screen display data of 
the partial screen display 502, the identification informa- 
tion of the cursor data of the cursor display element 601 
is transmitted to the device 200. The device 200 exe- 
cutes a predetermined function based on the identifica- 
tion information of the partial screen display data 
received as the operation request 241c (including the 
identification information of the cursor data) and the 
operation information. 

[0177] Then, when the state in the device 200 is 
changed and therefore the screen display data in the 
device 200 is changed, the device 200 spontaneously 
transmits the partial screen information 121b to the con- 
troller 1 00. When the controller 1 0Oissues the operation 
request 241c based on cursor move operation by the 
user, all that is changed on the operating screen is the 
position of the cursor display element. Therefore, only 
the partial screen information 121b including the cursor 
dispfay element information is transmitted to the control- 
ler 100. After receiving, based on the position informa- 
tion included in the cursor display element information, 
the controller 100 superimposes the cursor display ele- 
ment onto the operating screen for overlap display. 

<5. Fifth embodiment) 

[0178] Next, a network control system according to 
a fifth embodiment of the present invention is now 
described. The structure of the device and controller in 
the present embodiment is also similar to that in the first 
embodiment. Therefore, the same components are pro- 
vided with the same reference numerals, and their 
description is omitted herein (refer to FIGS. 1 and 2). 
[0179] FIG. 1 9 is a diagram showing the structure of 
screen information 55 sent from the device to the con- 
troller in the present embodiment. The screen informa- 
tion 55 is constructed of, as in the fourth embodiment, 
the screen information set list 71 , the main screen infor- 
mation 65, and cursor display element information 81 . 
In the present embodiment, the cursor display element 
is superimposed onto the operating screen for overlap 
display. In the screen information 55, the screen infor- 
mation set list 71 and the main screen information are 
similar in internal structure to those in the fourth embod- 
iment. The cursor display element information 81 is sim- 
ilar to that in the fourth embodiment in that a header 



includes an identifier, version information, and position 
information. However, instead of the cursor data as bit- 
map data indicating the cursor display element, the cur- 
sor display element information 81 has information 

5 indicative of the shape, size, and color of the cursor dis- 
play element, and further operation enable information. 
In this sense, the present embodiment is different from 
the fourth embodiment. Here, the operation enable 
information is information indicative of the type of oper- 

10 ation that the user is allowed to carry out. For example, 
the operation enable information indicates that the cur- 
sor can be moved from the present position only to 
upward and rightward directions among upward, down- 
ward, leftward, and rightward directions. 

15 [0180] FIG. 20 is a diagram showing one example 
of the operating screen displayed on the display screen 
of the controller in the present embodiment. In this 
example, the cursor display element 601 is superim- 
posed onto the operating screen itself for overlap dis- 

20 play. Here, assuming that the screen with the cursor 
display element 601 superimposed onto the screen dis- 
play 500 for overlap display is the operating screen in a 
broad sense, it can be considered that the screen dis- 
play data representing this broad operating screen 

25 (broad screen display data) is composed of two pieces 
of partial screen display data corresponding to the 
screen display 500 and the cursor display element 601 . 
Furthermore, although the cursor display element takes 
a rectangular shape in the example shown in FIG. 20, it 

30 may take other shapes such as circle. Note that, in the 
present embodiment, bitmap data indicative of the cur- 
sor display element does not exist, and the shape, size, 
and color of the cursor display element is determined 
according to information included in the cursor display 

35 element information 81 . 

[01 81 ] The protocol of the network control system in 
the present embodiment is basically similar to that in the 
second embodiment as shown in FIG. 13. In the present 
embodiment, the screen information 111b sent from the 

40 device 200 to the controller 100 as a response to the 
screen request 101 is structured as shown in FIG. 19. 
The main screen information 65 and the cursor display 
element information 81 correspond to different screen 
display information in the second embodiment. After 

45 receiving such screen information, the controller 100 
displays the operating screen on the display screen 
using the screen display data of the main screen infor- 
mation 65, and further superimposes the cursor display 
element onto the operating screen for overlap display 

so using the information about the shape, size, and color, 
and the position information in the cursor display ele- 
ment information 81 . 

[0182] After receiving the screen information 111b, 
when the operator carries out operation to the device 
55 through the operating screen as a user interface in the 
controller 1 00, the controller 1 00 transmits to the device 
200 the identification information of the screen display 
data indicative of the operating screen and the opera- 
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tion information indicative of the operation as the opera- 
tion request 241. Alternatively, before transmitting the 
operation request 241, the controller 100 may check 
whether the user operation is allowed or not based on 
the operation enable information in the cursor display 
element information 81 received as the screen informa- 
tion 1 1 1b. As a result of this check, if the operation is not 
allowed, the controller 100 does not transmit the opera- 
tion request 241 but carries out error handling, thereby 
preventing unnecessary processing. By way of exam- 
ple, if the cursor is located on contents 1 shown in FiG. 
20, the operation by the leftward direction arrow key on 
the remote controller may be made invalid. Alternatively, 
if the user presses or releases a selection key on the 
remote controller, not "press", "release", but "select" 
may be sent to the device as the user operation informa- 
tion. Therefore, the processing of the device can be sim- 
plified. 

[0183] After issuance of the operation request 241 , 
when the state in the device 200 is changed and there- 
fore the screen display data in the device 200 is 
changed, the device 200 spontaneously transmits to the 
controller 1 00 the partial screen information 121b corre- 
sponding to a changed part of the screen display data. 
In the present embodiment, if the cursor display ele- 
ment is changed by operation from the user, only the 
cursor display element information 81 as shown in FIG. 
19 is sent to the controller 100. After receiving, based 
on the position information and the information indica- 
tive of the shape, size, and color included in the cursor 
display element information 81, the controller 100 
superimposes the cursor display element onto the oper- 
ating screen for overlap display. 
[0184] As described above, according to the 
present embodiment, when the cursor display element 
is changed by operation from the user, the device does 
not have to send bitmap data such as screen display 
data to the controller, thereby improving the response 
speed to the user operation. Furthermore, by setting 
information indicative of the shape, size, and color 
included in the cursor display element information 81 , it 
is possible to conform the design of the cursor to the 
design of each display element on the operating screen. 

(Others) 

[0185] In each of the above described embodi- 
ments, units connected to the transmission path 1 are 
units handling at least one of video, audio, and informa- 
tion. However, units that do not directly relate to either 
video, audio, or information, such as air conditioner, 
may be connected to the transmission path 1 as devices 
to be controlled. The present invention can be applied to 
a network control system with such units connected 
thereto. 



INDUSTRIAL APPLICABILITY 

[01 36] The present invention is applied to a network 
control system in which a unit connected to a network is 

s operated through the network. Specifically, the present 
invention is suitable for a device control system using a 
graphical user interface (GUI) thai supports user's 
device operation through graphics, characters, etc., on 
a screen. Such system includes a network control sys- 

io tern in which a plurality of AV units are connected to 
each other a transmission path such as a serial bus of 
IEEE 1394 standard and one with a GUI controls the 
other units through the transmission path. 

is Claims 

1 . A network control system in which a first unit and a 
second unit are connected to each other through a 
transmission path and a controller included in said 

20 first unit controls a device in said second unit 
through said transmission path, 

at least one of said first unit and said second 
unit being for handling at least one of video, 
25 audio, and information, 

said device 

having screen display data for displaying 
an operating screen of said device and 
30 identification information for identifying the 

screen display data; and 
transmitting said screen display data and 
said identification information to said con- 
troller through said transmission path, and 

35 

said controller 

comprising a user interface including dis- 
play means; 

to receiving said screen display data and said 

identification information from said device 
through said transmission path; 
displaying said operating screen on said 
display means using said screen display 

45 data; and 

in response to an operation by a user to 
said operating screen, controlling said 
device by transmitting operation informa- 
tion indicative of the operation and said 

so identification information to said device 

through said transmission path. 

2. A network control system in which a first unit and a 
second unit are connected to each other a trans- 

55 mission path and a controller included in said first 
unit controls a device in said second unit through 
said transmission path, 
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47 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 
said device 

5 

having screen display data for displaying 
an operating screen of said device and 
identification information for identifying the 
screen display data; and 
transmitting said screen display data and to 
said identification information to said con- 
trailer through said transmission path, and 

said controller 

15 

comprising a user interface including dis- 
play means; 

receiving said screen display data and said 
identification information from said device 
through said transmission path; 20 
displaying said operating screen on said 
display means using said screen display 
data and said identification information; 6. 
and 

in response to an operation by a user to 2s 
said operating screen, controlling said 
device by transmitting operation informa- 
tion indicative of the operation and said 
identification information to said device 
through said transmission path. 30 

3. The network control system according to claim 1 or 
2, wherein 

said identification information includes version 3$ 
information indicating a version of said screen 
display data. 7. 

4. The network control system according to claim 1 or 

2, wherein 40 

said operation information includes an operat- 
ing position information indicative of an operat- 
ing position on said operating screen. 

45 

5. A network control system in which a first unit and a 
second unit are connected to each other a trans- 
mission path and a controller included in said first 
unit controls a device in said second unit through 
said transmission path, so 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 

said device 55 



displaying an operating screen of said 
device; and 

transmitting said partial screen display 
data to said controller through said trans- 
mission path, and 

said controller 

comprising a user interface including dis- 
play means; 

receiving said partial screen display data 
from said device through said transmission 
path; 

displaying said operating screen on said 
display means using said partial screen 
display data; and 

in response to an operation by a user to 
said operating screen, controlling said 
device by transmitting operation informa- 
tion indicative of the operation to said 
device through said transmission path. 

The network control system according to claim 5, 
wherein 

when said screen display data of said device is 
changed, said device transmits changed partial 
screen display data of said screen display data 
to said controller; and 

said controller receives the changed partial 
screen display data from said device through 
said transmission path and, based on the 
received partial screen display data, updates 
said operating screen displayed on said display 
means. 

The network control system according to claim 5, 
wherein 

said device 

has partial screen identification informa- 
tion for identifying said partial screen dis- 
play data; and 

transmits said partial screen display data 
and said partial screen identification infor- 
mation to said controller through said 
transmission path, and 

said controller 

receives said partial screen display data 
and said partial screen identification infor- 
mation from said device through said 
transmission path. 



having screen display data composed of a 8. The nelwork control system according to claim 7, 
plurality of partial screen display data for wherein 
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when said screen display data of said device is 
changed, said device transmits changed partial 
screen display data of said screen display data 
and the partial screen identification information 
of the partial screen display data to said con- s 
trailer, and 

said controller receives the changed partial 
screen display data and the partial screen 
identification information of the partial screen 
display data from said device through said 10 
transmission path and, based on the received 
partial screen display data and partial screen 
identification information, updates said operat- 
ing screen displayed on said display means. 

15 

9. The network control system according to claim 7, 
wherein 

in response to an operation by a user to said 
operating screen, said controller controls said 20 
device by transmitting operation information 
indicative of the operation and the partial 
screen identification information corresponding 
to the operation to said device through said 
transmission path. 25 

10. The network control system according to claim 7 or 
9, wherein 



11. The network control system according to claim 7 or 

9, wherein 35 

one display element in screen display is 
arranged in any one of a plurality of display 
parts corresponding to said plurality of partial 
screen display data. 40 

12. The network control system according to claim 7 or 
9, wherein 

a display element corresponding to each oper- 45 
ation by the user is arranged in any one of a 
plurality of display parts corresponding to said 
plurality of partial screen display data. 

13. A network control system in which a first unit and a so 
second unit are connected to each other a trans- 
mission path and a controller included in said first 
unit controls a device in said second unit through 
said transmission path, 

55 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 



said device 

having screen display data for displaying 
an operating screen of said device and 
overlap display data for overlap display on 
said operating screen; and 
transmitting said screen display data and 
said overlap display data to said controller 
through said transmission path, and 

said controller 

comprising a user interface including dis- 
play means; 

receiving said screen display data and said 
overlap display data from said device 
through said transmission path; 
displaying said operating screen on said 
display means using said screen display 
data, and carrying out overlap display on 
said operating screen displayed on said 
display means by using said overlap dis- 
play data; and 

in response to an operation by a user to 
said operating screen, controlling said 
device by transmitting operation informa- 
tion indicative of the operation to said 
device through said transmission path. 

14. The network control system according to claim 13, 
wherein 

said overlap display data is cursor information 
for displaying a cursor indicative of a position of 
operation by the user to said operating screen. 

15. The network control system according to claim 14, 
wherein 

said cursor information includes position infor- 
mation indicative of a position of said cursor on 
said operating screen. 

16. The network control system according to claim 14, 
wherein 

said cursor information includes shape infor- 
mation indicative of a shape of said cursor. 

17. The network control system according to claim 14, 
wherein 

said cursor information includes size informa- 
tion indicative of a size of said cursor. 

18. The network control system according to claim 14, 
wherein 



20 



25 



said partial screen identification Information 30 
includes version information indicating a ver- 
sion of said partial screen display data. 
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said cursor information includes color informa- 
tion indicative of a color of said cursor. 

19. The network control system according to claim 14, 
wherein 5 

said cursor information includes enable infor- 
mation indicating an operation that the user is 
allowed to perform. 

JO 

20. The network control system according to claim 13, 
wherein 



at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 
said device 

having screen display data for displaying 
an operating screen of said device and 
identification information for identifying the 
screen display data; 

transmitting said screen display data and 
said identification information to said con- 
troller through said transmission path, and 



when a display part corresponding to said over- 
lap display data of said device is changed, said 
device transmits said overlap display data to 
said controller, and 

said controller receives said overlap display 
data from said device through said transmis- 
sion path and, based on said received overlap 
display data, updates said operating screen 
displayed on said display means. 

21 . A network control system in which a first unit and a 
second unit are connected to each other a trans- 
mission path and a controller included in said first 
unit controls a device in said second unit through 
said transmission path, 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 
said device 

having screen display data for displaying 
an operating screen of said device and 
identification information for identifying the 
screen display data; and 
transmitting said screen display data and 
said identification information to said con- 
troller through said transmission path, and 

said controller 

comprising a user interface including dis- 
play means; 

receiving said screen display data and said 
identification information from said device 
through said transmission path; and 
displaying said operating screen on said 
display means using said screen display 
data. 

22. A network control system in which a first unit and a 
second unit are connected to each other a trans- 
mission path and a controller included in said first 
unit controls a device in said second unit through 
said transmission path, 



15 



20 



said controller 

comprising a user interface including dis- 
play means; 

receiving said screen display data and said 
identification information from said device 
through said transmission path; and 
displaying said operating screen on said 
display means using said screen display 
data and said identification information. 



25 23. A network control system in which a first unit and a 
second unit are connected to each other a trans- 
mission path and a controller included in said first 
unit controls a device in said second unit through 
said transmission path, 
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55 



at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 
said device 

having screen display data for displaying 
an operating screen of said device and 
overlap display data for overlap display on 
said operating screen; and 
transmitting said screen display data and 
said overlap display data to said controller 
through said transmission path, and 

said controller 

comprising a user interface including dis- 
play means; 

receiving said screen display data and said 
overlap display data from said device 
through said transmission path; and 
displaying said operating screen on said 
display means using said screen display 
data, and carrying out overlap display on 
said operating screen displayed on said 
display means by using said overlap dis- 
play data. 

24. A second unit that is connected through a transmis- 
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sion path to a first unit including a controller and 
includes a device controlled by said controller 
through said transmission path, 

at least one of said first unit and said second s 
unit being for handling at least one of video, 
audio, and information, 
said device 

having screen display data for displaying w 
an operating screen of said device and 
identification information for identifying the 
screen display data; 

transmitting said screen display data and 
said identification information to said con- is 
troll er through said transmission path; and 
receiving the identification information of 
the screen display data and operation 
information indicative of an operation by a 
user, and operating based on the received 20 
identification information and operation 
information. 

25. A first unit that is connected through a transmission 
path to a second unit and includes a controller for 25 
controlling a device included in said second unit 
through said transmission path, 

at least one of said first unit and said second 
unit being for handling at least one of video, ao 
audio, and information, 
said controller 

comprising a user interface including dis- 
play means; 35 
receiving screen display data indicative of 
an operating screen of said device and 
identification information for identifying the 
screen display data from said device 
through said transmission path; 40 
displaying said operating screen on said 
display means using said screen display 
data; and 

in response to an operation by a user to 
said operating screen, controlling said 45 
device by transmitting operation informa- 
tion indicative of the operation and said 
identification information to said device 
through said transmission path. 

50 

26. The second unit according to claim 24, wherein 

said operation information includes operating 
position information indicative of a position of 
operation on said operating screen. 55 

27. The first unit according to claim 25. wherein 



said operation information includes operating 
position information indicative of a position of 
operation on said operating screen. 

28. A second unit that is connected through a transmis- 
sion path to a first unit including a controller and 
includes a device controlled by said controller 
through said transmission path, 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 
said device 

having screen display data composed of a 
plurality of partial screen display data for 
displaying an operating screen of said 
device; 

transmitting said partial screen display 
data to said controller through said trans- 
mission path; and 

receiving operation information indicative 
of an operation by a user and operating 
based on the received operation informa- 
tion. 

29. A first unit that is connected through a transmission 
path to a second unit and includes a controller for 
controlling a device included in said second unit 
through said transmission path, 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 
said controller 

comprising a user interface including dis- 
play means; 

receiving a plurality of partial screen dis- 
play data indicative of an operating screen 
of said device through said transmission 
path from said device; 
displaying said operating screen on said 
display means using said partial screen 
display data; and 

in response to an operation by a user to 
said operating screen, controlling said 
device by transmitting operation informa- 
tion indicative of the operation through said 
transmission path to said device. 

3a The second unit according to claim 28, wherein 

when said screen display data of said device is 
changed, said device transmits changed partial 
screen display data of said screen display data 
to said controller. 
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31 . The first unit according to claim 29, wherein 

when said screen display data of said device is 
changed, said controller receives changed par- 
tial screen display data of said screen display 5 
data from said device through said transmis- 
sion path and, based on the received partial 
screen display data, updates said operating 
screen displayed on said display means. 

10 

32. A second unit that is connected through a transmis- 
sion path to a first unit including a controller and 
includes a device controlled by said controller 
through said transmission path, 

15 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, and 
said device 

20 

having screen display data for displaying 
an operating screen of said device and 
overlap display data for overlap display on 
said operating screen; 

transmitting said screen display data and 2s 
said overlap display data to said controller 
through said transmission path; and 
receiving operation information indicating 
a user's operation transmitted from said 
controller, and operating based on the 30 
received operation information. 

33. A first unit that is connected through a transmission 
path to a second unit and includes a controller for 
controlling a device included in said second unit 35 
through said transmission path, 

at least one of said first unit and said second 
unit being for handling at least one of video, 
audio, and information, 40 
said controller 

comprising a user interface including dis- 
play means; 

receiving screen display data indicative of 45 
an operating screen of said device and 
overlap display data for overlap display on 
said operating screen through said trans- 
mission path; 

displaying said operating screen of said so 
device on said display means using said 
screen display data, and performing over- 
lap display on said operating screen dis- 
played on said display means using said 
overlap display data; and 55 
in response to an operation by a user to 
said operating screen, controlling said 
device by transmitting operation informa- 



tion indicating the operation through said 
transmission path to said device. 

34. A control method, in a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path, at least one of said 
first unit and said second unit being for handling at 
least one of video, audio, and information, and a 
controller included in said first unit controlling a 
device included in said second unit through said 
transmission path, said control method comprising 
the steps of: 

transmitting screen display data for displaying 
an operating screen of said device and identifi- 
cation information for identifying the screen dis- 
play data from said device through said 
transmission path to said controller; 
displaying said operating screen on said con- 
troller using said screen display data transmit- 
ted from said device; and 
in response to an operation by a user to said 
operating screen, controlling said device by 
transmitting operation information indicative of 
the operation and said identification informa- 
tion from said controller through said transmis- 
sion path to said device. 

35. A control method, in a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path, at least one of said 
first unit and said second unit being for handling at 
least one of video, audio, and information, and a 
controller included in said first unit controlling a 
device included in said second unit through said 
transmission path, said control method comprising 
the steps of: 

transmitting screen display data for displaying 
an operating screen of said device and identifi- 
cation information for identifying the screen dis- 
play data from said device through said 
transmission path to said controller; 
displaying said operating screen on said con- 
troller using said screen display data and said 
identification information transmitted from said 
device; and 

in response to an operation by a user to said 
operating screen, controlling said device by 
transmitting operation information indicative of 
the operation and said identification informa- 
tion from said controller through said transmis- 
sion path to said device. 

36. A control method, in a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path, at least one of said 
first unit and said second unit being for handling at 
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least one of video, audio, and information, and a 
controller included in said first unit controlling a 
device included in said second unit through said 
transmission path, said control method comprising 
the steps of: 5 

transmitting at least one of plurality of partial 
screen display data composing screen display 
data for displaying an operating screen of said 
device from said device through said transmis- 10 
sion path to said controller; 
displaying said operating screen on said con- 
troller using said partial screen display data 
transmitted from said device; and 
in response to an operation by a user to said is 
operating screen, controlling said device by 
transmitting operation information indicative of 
fha operation from said controller through said 
« remission path to said device. 

20 

37. A control method, in a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path, at least one of said 
first unit and said second unit being for handling at 
least one of video, audio, and information, and a 25 
controller included in said first unit controlling a 
device included in said second unit through said 
transmission path, said control method comprising 
the steps of: 

30 

transmitting screen display data for displaying 
an operating screen of said device and overlap 
display data for overlap display on said operat- 
ing screen from said device through said trans- 
mission path to said controller; 35 
displaying said operating screen on said con- 
troller and performing overlap display on said 
displayed operating screen based on said over- 
lap display data by using said screen display 
data and said overlap display data transmitted 10 
from said device; and 

in response to an operation by a user to said 
operating screen, controlling said device by 
transmitting operation information indicative of 
the operation from said controller through said 45 
transmission path to said device. 

38. A control method, in a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path, at least one of said so 
first unit and said second unit being for handling at 
least one of video, audio, and information, and a 
controller included in said first unit controlling a 
device included in said second unit through said 
transmission path, said control method comprising 55 
the steps of: 

transmitting screen display data for displaying 



an operating screen of said device and identifi- 
cation information for identifying the screen dis- 
play data from said device through said 
transmission path to said controller; and 
displaying said operating screen on said con- 
troller using said screen display data transmit- 
ted from said device. 

39. A control method, in a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path, at least one of said 
first unit and said second unit being for handling at 
least one of video, audio, and information, and a 
controller included in said first unit controlling a 
device included in said second unit through said 
transmission path, said control method comprising 
the steps of: 

transmitting screen display data for displaying 
an operating screen of said device and identifi- 
cation information for identifying the screen dis- 
play data from said device through said 
transmission path to said controller; and 
displaying said operating screen on said con- 
troller using said screen display data and said 
identification information transmitted from said 
device. 

40. A control method, in a network control system in 
which a first unit and a second unit are connected to 
each other a transmission path, at least one of said 
first unit and said second unit being for handling at 
least one of video, audio, and information, and a 
controller included in said first unit controlling a 
device included in said second unit through said 
transmission path, said control method comprising 
the steps of: 

transmitting screen display data for displaying 
an operating screen of said device and overlap 
display data for overlap display on said operat- 
ing screen from said device through said trans- 
mission path to said controller; and 
displaying said operating screen on said con- 
troller and performing overlap display on said 
operating screen based on said overlap display 
data by using said screen display data and said 
overlap display data transmitted from said 
device. 
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